Progress!

Genuinely really impressive stuff from (some parts of) my old team at meta. Tactile sensing for manipulation has a ways to go but it's coming so far.

1/N Most Vision-Language-Action models need tons of data for finetuning, and still fail for new objects and instructions. Introducing OTTER, a lightweight, easy-to-train model that uses text-aware visual features to nail unseen tasks out of the box! Here's how it works 👇

📢🔥Excited to share that I have joined Amazon Robotics to work on some of the hardest real-world manipulation problems at unprecedented scale Looking forward to this new chapter after spending 5+ wonderful years at Meta FAIR leading advances in touch perception and dexterity

Very cool breakthrough by our physical AI and robotics teams-- Vulcan is the first robot that combines sight and touch, and can feel its way through cluttered spaces the way humans do. Vulcan is helping make work safer by handling ergonomically challenging tasks, while creating

Thrilled to be working on the Vulcan team. We have one of the first use cases in Amazon where contact-rich manipulation in clutter is necessary. Not many ways to go about this than to really embrace contact and leverage the sense of touch!

Anyone interested in tactile sensing for robotics should be following Akash's solid releases. How should we integrate rich tactile sensing modality for policy learning?

Magnetic-based tactile sensors offer a versatile solution for full-hand sensing. However, magnetic signals are not trivial to process to capture relevant contact information. Check out Sparsh-skin, general representations for magnetic skins!

Great work by Akash Sharma on training a single model across many tactile tasks! one model >> many different task specific models

Sparsh-skin, our next iteration of general pretrained touch representations Skin like tactile sensing is catching up on the prominent vision-based sensors with the explosion of new dexteorus hands A crucial step in leveraging full hand sensing; work led by Akash Sharma 🧵👇

Learning task-agnostic tactile representations is very valuable for dexterity! Check out this cool work by Akash Sharma that explores this while integrating the history of tactile information. This enables highly dexterous tasks—like plug insertion with a giant hand! 😁

Very interesting work! And great to see self-supervised learning being used for tactile data. This is critical to scaling tactile to the level that vision has scaled.

I'm a featured interview in our latest behind-the-scenes release! We break down the ML and perception that drives the whole-body manipulation behaviors from last year. It starts with a neat demo of Atlas's range-of-motion and our vision foundation models. youtu.be/oe1dke3Cf7I?si…

Everyday human data is robotics’ answer to internet-scale tokens. But how can robots learn to feel—just from videos?📹 Introducing FeelTheForce (FTF): force-sensitive manipulation policies learned from natural human interactions🖐️🤖 👉 feel-the-force-ftf.github.io 1/n

📢Life is a sequence of bets – and I’ve picked my next: myolab.ai It’s incredibly ambitious, comes with high risk, & carries unbounded potential. But it’s a version of the #future I deeply believe in. I believe: ➡️AI will align strongly with humanity - coz it maximizes its own

I'll be speaking at the RSS Dexterous Manipulation Workshop tomorrow, discussing our recent work with Atlas!

So you’ve trained your favorite diffusion/flow based policy, but it’s just not good enough 0-shot. Worry not, in our new work DSRL - we show how to *steer* pre-trained diffusion policies with off-policy RL, improving behavior efficiently enough for direct training in the real

We demo’d Amazon grasp model at RSS this year. We performed over 600 grasps over one day at roughly 80-90% SR 1. On an open item set (people gave random often adversarial items), 2. In a random scene fully outdoor throughout the day 3. On a new embodiment (different from