you can just build things

A new VLA for navigation that can take in goal images, positions, and language, and exhibits some pretty neat emergent language following!

📢The next milestone for intelligent general-purpose robots has arrived! Announcing Gemini Robotics 1.5, our flagship system which brings breakthroughs from frontier models to the physical world with two new SOTA generalists: the GR 1.5 VLA and GR 1.5 embodied reasoning model 🧵

Please join us at Humanoids next week, October 2nd !! ❤️🤖❤️🎨

😠💢😵‍💫Tired of endless data collection & fine-tuning every time you try out VLA? Meet RDT2, the first foundation model that zero-shot deploys on any robot arms with unseen scenes, objects & instructions. No collection. No tuning. Just plug and play🚀 Witness a clear sign of

Tired of collecting robot demos? 🚀 Introducing CP-Gen: geometry-aware data generation for robot learning. From a single demo, CP-Gen generates thousands of new demonstrations to train visuomotor policies that transfer zero-shot sim-to-real across novel geometries and poses.

“…turns out the ‘render’ subset of simulation is well-matched to the ‘quasi-static’ subset of manipulation.”

Come check out Pranav's presentation on RoboArena, a framework for evaluating generalist robot policies (like VLAs) with a distributed network of evaluators.

Robotic manipulation has seen tremendous progress in recent years but rigorous evaluation of robot policies remains a challenge! We present our work: "Reliable and Scalable Robot Policy Evaluation with Imperfect Simulators"! 🧵

Most VLAs have no memory. MemER is a scalable method for giving VLAs long-term visual memory. Key ideas: - relevant info is in a small # of frames: store just a few key frames - manage & use memory at high level of abstraction, with a HL policy Paper: jen-pan.github.io/memer/

Had a blast visiting CMU Robotics Institute and gave a talk at the RI Seminar today, where I briefly mentioned our new work, PLD, on self-improving VLAs. It achieved 99.2% on LIBERO and a one-hour continuous execution of GPU assembly with a 100% success rate. Check this out!

In my opinion, VLAs research is extremely empirical compared to many other directions. Simulations like LIBERO are no longer statistically meaningful to VLAs, as we can overfit to ~99% easily now. Urgent priorities: 1) Create new sim benchmarks 2) Show real-world experiments

Just read through the new LeJEPA paper by Yann LeCun and Randall Balestriero. I’ve been curious to know what Yann’s been working on lately, especially considering all his criticisms of LLMs (which I disagree with, as I think LLMs will keep improving and will take us to ASI fairly

Today, we present a step-change in robotic AI Sunday. Introducing ACT-1: A frontier robot foundation model trained on zero robot data. - Ultra long-horizon tasks - Zero-shot generalization - Advanced dexterity 🧵->

Open-source: complete codebase covering multiple simulation backends, training, retargeting, and real-world inference. Infra built for humanoid, but also readily modified for quadruped (also included). Lots of infra gems/conveniences we rely on consistently. Hopefully equally

I have and will continue beating this drum- sim is the way. sim scales so much better than real. by the time we exhaust sim capabilities today, sim will have evolved to handle all the parts that are currently hard (contacts, soft bodies, etc.)

1/N 🎨🤖Given only a static image of an oil painting by an expert artist, can a robot infer the corresponding control actions, such as trajectory, orientation, and applied force, to accurately reproduce the painting? 🖌️Introducing IMPASTO: a robotic oil-painting system that

All of my Deep RL course lecture videos from Spring 2025 are now online! 🥳 Youtube playlist: youtube.com/watch?v=EvHRQh…