This spring I am teaching a new class at MIT called **How to AI (Almost) Anything** Its name is a play on 2 seminal MIT Media Lab courses: how to make almost anything (on design & fabrication) and how to grow almost anything (on synthetic biology) We are now in the AI age, and

What makes a Large Language Model unique? I am excited to share our new work “Idiosyncrasies in Large Language Models”. We demonstrate that LLMs exhibit idiosyncrasies – unique patterns in their outputs that enable us to distinguish these models with exceedingly high accuracies.

Interacting with the external world and reacting based on outcomes are crucial capabilities of agentic systems, but existing LLMs’ ability to do so is limited. Introducing Paprika 🌶️, our work on making LLMs general decision makers than can solve new tasks zero-shot. 🧵 1/n

1/Being in academia is such a privilege: You get to collaborate with insanely talented & passionate students on their journey to upskill themselves. Very excited to share *OpenUnlearning*: a unified, easily extensible framework for unlearning led by Anmol Mekala Vineeth🧵

📉📉NEW SCALING LAW PHENOMENON 📉📉 We find that knowledge and reasoning exhibit different scaling behaviors! Super excited to finally tell you all about our paper on the compute optimal scaling of skills: arxiv.org/pdf/2503.10061 [1/n]

Are you a frontier lab investing untold sums in training? Are you trying to stay competitive? Are you finding that your competitors' models are ... thinking a bit too much like yours? Then antidistillation.com might be for you! Sam Altman Elon Musk

Internet-scale datasets of videos and natural language are a rich training source! But can they be used to facilitate novel downstream robotic behaviors across embodiments and environments? Our new #ICLR2025 paper, Adapt2Act, shows how.

Why do larger language models generalize better? In our new ICLR paper, we derive an interpretable generalization bound showing that compute-optimal LLMs provably generalize better with scale! 📄arxiv.org/abs/2504.15208 1/7🧵

✨ Love 4o-style image generation but prefer to use Midjourney? Tired of manual prompt crafting from inspo images? PRISM to the rescue! 🖼️→📝→🖼️ We automate black-box prompt engineering—no training, no embeddings, just accurate, readable prompts from your inspo images! 1/🧵

Why can foundation models transfer to so many downstream tasks? Will the scaling law end? Will pretraining end like Ilya Sutskever predicted? My PhD thesis builds the contexture theory to answer the above. Blog: runtianzhai.com/thesis Paper: arxiv.org/abs/2504.19792 🧵1/12

🚀 Excited to share our new work on data generation for IR! We create synthetic multi-level ranking contexts for training dense retrievers. Now it’s easy to build custom retrieval datasets and move beyond the standard InfoNCE loss by learning from fine-grained relevance levels!🧵

In our #AISTATS2025 paper, we ask: when it is possible to recover a consistent joint distribution from conditionals? We propose path consistency and autoregressive path consistency—necessary and easily verifiable conditions. See you at Poster session 3, Monday 5th May.

Introducing COMPACT: COMPositional Atomic-to-complex Visual Capability Tuning, a data-efficient approach to improve multimodal models on complex visual tasks without scaling data volume. 📦 arxiv.org/abs/2504.21850 1/10

Excited to share our work with my amazing collaborators, Goodeat, Xingjian Bai, Zico Kolter, and Kaiming. In a word, we show an “identity learning” approach for generative modeling, by relating the instantaneous/average velocity in an identity. The resulting model,

Big news! 🎉 I’m joining UNC-Chapel Hill as an Assistant Professor in Computer Science starting next year! Before that, I’ll be spending time OpenAI working on LLM privacy. UNC Computer Science UNC NLP

✨ Did you know that NOT using all generated rollouts in GRPO can boost your reasoning LLM? Meet PODS! We down-sample rollouts and train on just a fraction, delivering notable gains over vanilla GRPO. (1/7)

🥳🥳🥳I defended my PhD thesis today! Special thanks to my wonderful advisor Zico Kolter and committee members Russ Salakhutdinov Graham Neubig Lester Mackey! 🎉🎉🎉I am joining OpenAI as a researcher, super excited to keep working on frontier models and meet everyone in SF!