What design choices matter when developing a visually-conditioned language model (VLM)? Check out our paper – Prismatic VLMs – and open-source training code, evaluation suite, and 42 pretrained VLMs at the 7B-13B scale! 📜 arxiv.org/abs/2402.07865 ⚙️ + 🤗 github.com/TRI-ML/prismat…

we just released the sora research blog with additional details and results. check it out! openai.com/research/video…

Can we collect robot data without any robots? Introducing Universal Manipulation Interface (UMI) An open-source $400 system from Stanford University designed to democratize robot data collection 0 teleop -> autonomously wash dishes (precise), toss (dynamic), and fold clothes (bimanual)

# on technical accessibility One interesting observation I think back to often: - when I first published the micrograd repo, it got some traction on GitHub but then somewhat stagnated and it didn't seem that people cared much. - then I made the video building it from scratch,

Join us in discussing the future of foundation models and AI for general-purpose robotics at ICRA 2024! Deadline for the submitting to Vision-Language Models for Navigation and Manipulation workshop is in less than two weeks!

Cooking in kitchens is fun. BUT doing it collaboratively with two robots is even more satisfying! We introduce MOSAIC, a modular framework that coordinates multiple robots to closely collaborate and cook with humans via natural language interaction and a repository of skills.

Cross-embodied robot policies hold the promise of one policy to control all robots. But how far does transfer go? In new work, we study positive transfer between *manipulation* & *navigation* and show that nav data helps manipulation, and vice versa! extreme-cross-embodiment.github.io 🧵 👇

After 3 years @tesla and Optimus, I am thrilled to announce that I joined Hugging Face to start an ambitious open robotics project! (open as in open-source, not as in Open AI) Looking for engineers to build real robots in Paris 🇫🇷 apply.workable.com/huggingface/j/… 🤖🤗

I’m supposed to be graduating this week 🎓 But instead, I’ll be at #ICRA2024 in beautiful Japan all week, presenting an award finalist talk on Tue and at a workshop on Friday! Come find me / DM to chat all things robot learning, job market, veggie food @ Japan and karaoke 🍜🍵🎤

We’ll be presenting NoMaD today at the Awards track at #ICRA2024, where it’s nominated for 3 Best Paper awards!! If you missed the talk yesterday, come find Ajay Sridhar and me at the 13:30 poster session. Good luck navigating the maze

Cross embodiment for manipulation (RT-X) and cross embodiment for navigation (NoMAD) win best paper at #ICRA2024 Big congratulations in order for my colleagues and students, congratulations! Seems pretty clear where the field is headed...

Are large vision action models THE MOST important research topic in robotics for the next 5 years? 50-50 split in the room here. Fight!

I decided to try my hand at blogging during my sabbatical. First up: "I am your PhD advisor: but what does that actually mean?" On the importance of setting clear expectations in the advisor-advisee relationship, and my version of these in written form. open.substack.com/pub/animajumda…

🚨Important update from our Robot Learning Lab in London. Following recent news, we’re moving on after a wonderful 2 years… Today, we unveil 4 big pieces of research from our incredible team. Check out the compilation video and thread below to see our final work! 📽️👇

How can Gemini 1.5 Pro’s long context window help robots navigate the world? 🤖 A thread of our latest experiments. 🧵

I “defended” my thesis earlier today — super grateful to Sergey Levine and everyone at Berkeley AI Research for their support through the last 5 years! 🐻 Excited to be joining Google DeepMind and continue the quest for bigger, better, smarter robot brains 🤖🧠

The countdown for ICLR 2025 starts today! Timely with NeurIPS Conference rebuttals underway. The PC ICLR 2025 this year is introducing a new Reciprocal Reviewing Requirement - All authors (individual on >=3 papers) must individually provide service as a reviewer for >=6 papers (or

We had an amazing time doing live demos at CoRL last year, definitely recommend signing up. Your robots will thank you 😊