1X World Model Scaling Evaluation for Robots

Excited to share more of what I've been working on for while now! We've trained a world model to solve policy evaluation, where we see enough correlation to real-world evaluations for practical production usage.

Today we're excited to share a glimpse of what we're building at Generalist. As a first step towards our mission of making general-purpose robots a reality, we're pushing the frontiers of what end-to-end AI models can achieve in the real world. Here's a preview of our early

Scaling action prediction with actionfree data is the challenge that we have on our path to scaling robotics. We can reframe the problem in a "what" and "how" framework. This allows building large scale models trained on different subsets of data with (and without) action

We’re releasing the RoboArena today!🤖🦾 Fair & scalable evaluation is a major bottleneck for research on generalist policies. We’re hoping that RoboArena can help! We provide data, model code & sim evals for debugging! Submit your policies today and join the leaderboard! :) 🧵

We’re bringing powerful AI directly onto robots with Gemini Robotics On-Device. 🤖 It’s our first vision-language-action model to help make robots faster, highly efficient, and adaptable to new tasks and environments - without needing a constant internet connection. 🧵

3-months ago we went from only driving in the UK, California and Germany, to driving in seven different countries with very different climates. 🌍 🇺🇸 In the US, we went from driving in one state to driving in ten! From the sunny streets of California to the snowy mountain

Google DeepMind announces Gemini Robotics On-Device - an efficient VLA model optimized to run locally with low-latency inference. It enables general-purpose dexterity, adapts to new tasks or robot hardware with fewer than 100 demos.

Founder and CEO of The Bot Company Kyle Vogt—who also cofounded Twitch and Cruise—shares a pint and discusses robots doing your household chores, the autonomy industry, Tesla vs Waymo, and how future $100b companies will look different. Timestamps: 00:00 Intro 00:38 The Bot Company

If you have a policy that uses diffusion/flow (e.g. diffusion VLA), you can run RL where the actor chooses the noise, which is then denoised by the policy to produce an action. This method, which we call diffusion steering (DSRL), leads to a remarkably efficient RL method! 🧵👇

Hello Roboticists, Trossen Robots are now live on Open GRID. With just a few lines of code, any Trossen arm — old or new — can tap into AI skills from the cloud. One example: Teleoperation via API → data collection on Isaac Sim → playback on real hardware 👇🏻 Making Trossen

What would a World Model look like if we start from a real embodied agent acting in the real world? It has to have: 1) A real, physically grounded and complex action space—not just abstract control signals. 2) Diverse, real-life scenarios and activities. Or in short: It has to

If you’re working on robotics and AI, the recent Stanford talk from Russ Tedrake on scaling multitask robot manipulation is a mandatory watch, full stop. No marketing, no hype. Just solid hypothesis driven science, evidence backed claims. A gold mine in today’s landscape!

DreamVLA A Vision-Language-Action Model Dreamed with Comprehensive World Knowledge

Wrote a new post about Flow Matching, where I try to explain my intuitions about it as concisely as possible. Hope it's useful :)

Large robot datasets are crucial for training 🤖foundation models. Yet, we lack systematic understanding of what data matters. Introducing MimicLabs ✅System to generate large synthetic robot 🦾 datasets ✅Data-composition study 🗄️ on how to collect and use large datasets 🧵1/

VLMs often struggle with physical reasoning tasks such as spatial reasoning. Excited to share how we can use world models + test-time search to zero-shot improve spatial reasoning in VLMs!

I wrote a fun little article about all the ways to dodge the need for real-world robot data. I think it has a cute title. sergeylevine.substack.com/p/sporks-of-agi

GR-3 Technical Report

I'm observing a mini Moravec's paradox within robotics: gymnastics that are difficult for humans are much easier for robots than "unsexy" tasks like cooking, cleaning, and assembling. It leads to a cognitive dissonance for people outside the field, "so, robots can parkour &