Dense language annotations will enable us to do more with a given dataset in robotics, just as in image/video settings. The effectiveness of this approach will only improve as foundation models get better at producing detailed language labels given behavior clips.

We found using an exoskeleton based approach to be much more effective than VR for teleop. Likely due to higher accuracy and lower latency since this doesn’t rely on vision estimation of hand positions, instead directly commanding robot joint positions.

These hands balance high controllability, reactivity and power, without blowing up design size. Very exciting ! In personal news, I recently completed my PhD from CMU, and have joined Optimus AI ! Would like to thank Deepak Pathak for his guidance and mentorship.

Was really rewarding pushing on getting Optimus to confidently walk outside the lab on challenging terrain, in my first couple of months! I am blown away by the drive, ambition, vision and resources of the team Tesla Optimus. Come join us to make robot learning sci-fi dreams a

Tesla is where real-world AI is happening. These runs are on mulched ground, where I’ve myself slipped before. What’s really crazy here is that for these, Optimus is actually blind! Keeping its balance without video (yet), only other on-board sensors consumed by a neural net

I forgot to mention that Optimus was doing this while “blindfolded”. We haven’t yet made vision part of the control loop for uneven terrain. Optimus robots are regularly cruising around our workplace in Palo Alto where vision is used for static & dynamic object avoidance.

1.5 yrs ago, we set out to answer a seemingly simple question: what are we *actually* getting out of RL in fine-tuning? I'm thrilled to share a pearl we found on the deepest dive of my PhD: the value of RL in RLHF seems to come from *generation-verification gaps*. Get ready to🤿!

We're continuing to leverage RL for Optimus, coming soon for manipulation tasks as well! Come join us to build towards generalist robots, utilizing simulation and videos.

Really impressive generalization results, more evidence that data scaling with powerful ML models will work for robotics.

Early steps in building a legion for the next level of data scaling !

Agility with RL!

The sim2real RL pipeline we’ve built scales to many different challenging motions. This tech will soon be extended to useful manipulation tasks.

One of our goals is to have Optimus learn straight from internet videos of humans doing tasks. Those are often 3rd person views captured by random cameras etc. We recently had a significant breakthrough along that journey, and can now transfer a big chunk of the learning

The biggest product ever

The exponential increase in data will lead to a step change for general AI for applications beyond robotics. Pre-training datasets will not be limited to the internet.

For the past several days, Tesla has been testing self-driving Model Y cars (no one in driver’s seat) on Austin public streets with no incidents. A month ahead of schedule. Next month, first self-delivery from factory to customer.

BREAKING: First ever Tesla Model Y robotaxi with no-one in the drivers seat spotted testing on public roads in Austin, Texas! Tesla's new "Robotaxi" wordmark/logo is on the side of the vehicle.

Interesting work on learning more efficiently from demos. Learns a world models that allows search at inference time.

Super congratulations to the Tesla AI software & chip design teams on a successful @Robotaxi launch!! Culmination of a decade of hard work. Both the AI chip and software teams were built from scratch within Tesla.