Additionally, looking towards the future, we’re releasing PARTNR: a benchmark for Planning And Reasoning Tasks in humaN-Robot collaboration. Built on Habitat 3.0, it’s the largest benchmark of its kind to study and evaluate human-robot collaboration in household activities By

[NeurIPS D&B Oral] Embodied Agent Interface: Benchmarking LLMs for Embodied Agents A single line of code to evaluate your model! 🌟Standardize Goal Specifications: LTL 🌟Standardize Modules and Interfaces: 4 modules, 438 tasks, 1475 goals 🌟Standardize Fine-grained Metrics: 18

🤖 Introducing Human-Object Interaction from Human-Level Instructions! First complete system that generates physically plausible, long-horizon human-object interactions with finger motions in contextual environments, driven by human-level instructions. 🔍 Our approach: - LLMs

This is the mobile manipulation I want to see. You can only get this via RL.

The trained policy can be integrated with a high-level planner for real-world applications. By combining our object manipulation policy with user commands, we demonstrate its effectiveness in real-world scenarios—such as moving large trash carts. (6/8)

🪑How do you train robots to move furniture? This requires robots to synchronize whole-body movements, making teleoperation or RL approaches challenging. Check out this amazing work by Tianyu Li EasyPaperSniper, using human demonstrations to train robots to move furniture in the real world!

Meta PARTNR is a benchmark for planning and reasoning in embodied multi-agent tasks. This large-scale human and robot collaboration benchmark was core to our recent demos and also informs our work as scientists and engineers pushing this field of study forward.

How to achieve human-level open-ended machine Theory of Mind? Introducing #AutoToM: a fully automated and open-ended ToM reasoning method combining the flexibility of LLMs with the robustness of Bayesian inverse planning, achieving SOTA results across five benchmarks. 🧵[1/n]

🚨New Preprint 🚨 Embodied agents 🤖 operating in indoor environments must interpret ambiguous and under-specified human instructions. A capable household robot 🤖 should recognize ambiguity and ask relevant clarification questions to infer the user🧑‍🚒 intent accurately, leading

How do we enable agents to perform tasks even when these are underspecified? In this work, led by Ram Ramrakhya, we train VLA agents via RL to decide when to act in the environment or ask clarifying questions, enabling them to handle ambiguous instructions ram81.github.io/projects/ask-t…

I will be at ICLR to present PARTNR. Reach out if you want to talk about our work at FAIR or interesting problems in Robotics!

DexMachina lets us perform a functional comparison between different dexterous hands: we evaluate 6 hands on 4 challenging long-horizon tasks, and found that larger, fully actuated hands learn better and faster, and high DoF is more important than having human-like hand sizes –

I'll be giving two talks at the #CVPR2025 workshops: 3D LLM/VLA 3d-llm-vla.github.io and POETS poets2024.github.io/poets2025/. 🧵

I will be talking at the #CVPR2025 workshop on Humanoid Agents, tomorrow June 11th at 9:30 am. I will discuss how humanoid agents can help us improve human-robot collaboration. See you there! humanoid-agents.github.io

🚀 Excited to introduce SimWorld: an embodied simulator for infinite photorealistic world generation 🏙️ populated with diverse agents 🤖 If you are at #CVPR2025, come check out the live demo 👇 Jun 14, 12:00-1:00 pm at JHU booth, ExHall B Jun 15, 10:30 am-12:30 pm, #7, ExHall B

🤖 Does VLA models really listen to language instructions? Maybe not 👀 🚀 Introducing our RSS paper: CodeDiffuser -- using VLM-generated code to bridge the gap between **high-level language** and **low-level visuomotor policy** 🎮 Try the live demo: robopil.github.io/code-diffuser/ (1/9)

Check out our workshop on Continual Robot Learning from Humans, at #RSS2025, with amazing speakers covering topics including learning from human visual demonstrations, generative models for continual robot learning or the role of LLMs in embodied contexts …-robot-learning-from-humans.github.io