World model is the path towards building general intelligent agents

I was kidding -- this video was entirely simulated by the _world model_ we're building. 😀 It's mind-blowing how it produces high-fidelity simulations, lasting several minutes, to complete non-trivial tasks. This showcases the potential for infinite data & experience in

Alright starting to buy into the world models hype

Technology has advanced so much that I can't distinguish between autonomous robot, video generation, and game engine

They are all use cases of world models:)

Zhiting Hu Chen Tessler C Zhang World model neural sims definitely are very diverse in terms of what can be simulated. It is only bounded by the data it ingests which flexibly can include the worlds video data. Sim accuracy heavily depends on many factors, particularly on what you are simulating to begin with.

Andrej Karpathy Reminds me of the ideas behind this paper: arxiv.org/abs/2205.12548

A humanoid robot dancing with agility and flair💃 ... in a world _interactively_ simulated by world model Here’s the choreography we told the model to simulate, step by step: 💃Wave both arms and start jumping 👋 💃Dance dance dance‼️ 💃Stand still and put left arm

🔥🕹️Incredible fluidity and expressive motion -- beautifully captured by the world model

From human reasoning to natural evolution, the universe works by simulating possibilities recursively and allowing complexity to emerge. World model aim to characterize this core mechanism of simulation. Excited to be working on PAN World Model, a new foundation model for

We've been wondering about these too and studied multi-domain RLVR! One finding suggests that the conclusion "RL only elicits pretrained knowledge" is nuanced and varies by domain: 🔥 Heavily pretrained domains (Math, Code, Science) are indeed more readily "elicited." They

🚀 Excited to introduce SimWorld: an embodied simulator for infinite photorealistic world generation 🏙️ populated with diverse agents 🤖 If you are at #CVPR2025, come check out the live demo 👇 Jun 14, 12:00-1:00 pm at JHU booth, ExHall B Jun 15, 10:30 am-12:30 pm, #7, ExHall B

Thrilled to finally share SimWorld — the result of over a year’s work of the team. Simulators have been foundational for embodied AI research (I’ve worked with AI2Thor, CARLA, Genesis…), and SimWorld pushes this further with photorealistic Unreal-based rendering and scalable

A preview of SimWorld 🌏🌆🏙️ An Open-ended Simulator for Agents in Physical and Social Worlds Check out #CVPR2025 live demo today at 10:30am, #7, ExHall B

🕹️LIVE at #CVPR2025 — Today 10:30AM | Demo #7 | ExHall B Come see and play SimWorld in action! A large-scale, Unreal-powered simulator for training & evaluating LLM/VLM agents in open-ended physical & social environments 🌍🤖 🎥 Demo video: youtu.be/H_BoR3T59iA

📢We show that continuous latent reasoning has a theoretical advantage over discrete token reasoning (arxiv.org/abs/2505.12514): For a graph with n vertices and graph diameter D, a two-layer transformer with D steps of continuous CoTs can solve the directed graph reachability

Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective "We introduce GURU, a curated RL reasoning corpus of 92K verifiable examples spanning six reasoning domains—Math, Code, Science, Logic, Simulation, and Tabular—each built through domain-specific

Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective

🔥Reinforcement learning for LLM reasoning is emerging—but many questions remain🧐🧐 ❓ Does RL teach new reasoning, or just elicit what’s already in the base LLM? ❓ Do long chains of thought truly emerge from RL? ❓ Most RL work has been focusing on math and coding. But how do

Thanks for the suggestion! We actually tried Llama-3.1-8B, but found high "instruction-following costs" - the base model struggles with both task-specific prompts (e.g., outputting nested lists in CodeIO) and system prompts requiring special tokens like <think>, which is