Our first attempt at learning an artificial visual cortex for embodied agents. Our largest pre-trained model trained on our biggest curated dataset (VC-1) is competitive with or outperforms the best prior task-specific results on all benchmarks that we test on.

Extremely excited to share our new work led by Gunshi Gupta and Karmesh Yadav showing that pretrained diffusion models provide powerful vision-language representations for control tasks that drive efficiency and generalization! All code open-sourced at: github.com/ykarmesh/stabl…

📢Check out our paper ++ Pre-trained Text-to-Image Diffusion Models Are Versatile Representation Learners for Control ++ 📄arxiv.org/abs/2405.05852 The paper was highlighted as a contributed talk at the #GenAI4DM Workshop at #ICLR2024! Join us on Saturday, 11:30am at Lehar 3.

This paper will be presented as a **Spotlight Talk** at #NeurIPS2024! 🚀🤖 Congratulations to the student lead authors Gunshi Gupta and Karmesh Yadav! 🎉 Check out the thread🧵below!

Thrilled to have contributed to this work, now published in Nature! Check out Microsoft’s blog for live-action gameplay sequences from Muse: microsoft.com/en-us/research…

Excited to give a talk at the VPLow Workshop @ CVPR today (10am CT)! I’ll share our recent work on building benchmarks + methods to help multimodal agents remember better over long horizons. Tune in if you're interested in memory & open-world learning! 🔗 vplow.github.io/vplow_5th.html