🚨 Are you using empirical benchmarks to evaluate your LLM unlearning method? Our new paper arxiv.org/pdf/2410.02879 investigates how success on these benchmarks can be misleading. A🧵: 1/n

A dream I've had for five years is finally coming true: I'll be co-teaching a course next sem. on the algorithmic foundations of imitation learning / RLHF with my advisors, Drew Bagnell and Steven Wu! Sign up if you're at CMU (17-740) or follow along at interactive-learning-algos.github.io!

We’ve released updated 2024 lectures from my Stanford RL CS234 class! shorturl.at/pf427 This includes a guest lecture on Direct Preference Optimization (DPO) from 1st authors Rafael Rafailov @ NeurIPS Archit Sharma Eric tinyurl.com/3kr4czth Thanks to Stanford Online!

If you'd like to avoid the bends, the TL;DR is that RL lets us filter down our search space to only those policies that are optimal for relatively simple verifiers. 📰: arxiv.org/abs/2503.01067. Joint w/ the all-star cast of Sanjiban Choudhury, Wen Sun, Steven Wu, and Drew Bagnell. [2/n]

I was lucky enough to be invited give a talk on our new paper on the value of RL in fine-tuning at Cornell University last week! Because of my poor time management skills, the talk isn't as polished as I'd like, but I think the "vibes" are accurate enough to share: youtu.be/E4b3cSirpsg.

blog.ml.cmu.edu/2025/04/18/llm… 📈⚠️ Is your LLM unlearning benchmark measuring what you think it is? In a new blog post authored by Pratiksha Thaker, Shengyuan Hu, Neil Kale, Yash Maurya, Steven Wu, and Virginia Smith, we discuss why empirical benchmarks are necessary but not

blog.ml.cmu.edu/2025/05/22/unl… Are your LLMs truly forgetting unwanted data?  In this new blog post authored by Shengyuan Hu, Yiwei Fu, Steven Wu, and Virginia Smith, we discuss how benign relearning can jog unlearned LLM's memory to recover knowledge that is supposed to be forgotten.

Young scientists: Hang in there. The world needs you. I'm working on a solution

blog.ml.cmu.edu/2025/06/01/rlh… In this in-depth coding tutorial, Zhaolin Gao and Gokul Swamy walk through the steps to train an LLM via RL from Human Feedback!

Say ahoy to 𝚂𝙰𝙸𝙻𝙾𝚁⛵: a new paradigm of *learning to search* from demonstrations, enabling test-time reasoning about how to recover from mistakes w/o any additional human feedback! 𝚂𝙰𝙸𝙻𝙾𝚁 ⛵ out-performs Diffusion Policies trained via behavioral cloning on 5-10x data!

It was a dream come true to teach the course I wish existed at the start of my PhD. We built up the algorithmic foundations of modern-day RL, imitation learning, and RLHF, going deeper than the usual "grab bag of tricks". All 25 lectures + 150 pages of notes are now public! 🧵

ICML's election for their board of directors has begun. I've thrown my hat in the ring. Please consider voting for Gautam Kamath. I have experience with the governance of TMLR, COLT, and ALT, and I think I've demonstrated myself as a consciencious and engaged community member.

Recent work has seemed somewhat magical: how can RL with *random* rewards make LLMs reason? We pull back the curtain on these claims and find out this unexpected behavior hinges on the inclusion of certain *heuristics* in the RL algorithm. Our blog post: tinyurl.com/heuristics-con…

There are many great researchers out there. But the ones that really stand out to me are the ones who are also kind, even when they don't need to be.

Tu Youyou became the first mainland Chinese scientist to be awarded a #NobelPrize in a scientific field - for discovering artemisinin, a malaria cure that’s saved millions. Today we reveal the 2025 medicine laureate. Stay tuned.

The FORC 2026 call for papers is out! responsiblecomputing.org/forc-2026-call… Two reviewing cycles with two deadlines: Nov 11 and Feb 17. If you haven't been, FORC is a great venue for theoretical work in "responsible AI" --- fairness, privacy, social choice, CS&Law, explainability, etc.

A subtle aspect of predictive modeling is target variable construction: translating an unobservable concept like "healthcare need" into a prediction target But how does target variable construction unfold in practice, and how can we better support it going forward? #CSCW2025🧵

As good a time as any to announce I'm on the job market this year! I develop provably efficient reinforcement learning algorithms that are directly applicable to problems across both robotics and language modeling. See gokul.dev for more!

I was, until last year, on an H1B visa. And it’s not that US citizens don’t want to do my job - it’s that I don’t know any US citizen willing to subject themselves to be a postdoc for over nine years, like I was. US citizens will not fill these positions.

I'm teaching a new "Intro to Modern AI" course at CMU this Spring: modernaicourse.org. It's an early-undergrad course on how to build a chatbot from scratch (well, from PyTorch). The course name has bothered some people – "AI" usually means something much broader in academic