This project started with us annoyed at papers evaluating CoT "reasoning" with only GSM8k & MATH. We didn't expect to find such strong evidence that these are the only type of problem where CoT helps! Credit to Juan Diego Rodríguez (he/him) & Kyle Mahowald for driving the rigorous meta-analysis!

Our cross-university(s) collaborative work on "Scaling laws for Reward Model Overoptimization in Direct Alignment Algorithms" is accepted at NeurIPS Conference!

"Predicting Future Actions of Reinforcement Learning Agents" - Chung et al. We introduce the problem of predicting RL agents' behavior, which could have important safety implications. We find that RL agents that perform explicit (or implicit) planning can be more predictable.

For those interested, the keynotes of the RL_Conference 2024 are now available online: youtube.com/@RL-conference… Unfortunately, Doina Precup's talk was not recorded, but we have: Andy Barto, Emma Brunskill, Finale Doshi-Velez, Sergey Levine, David Silver, and Peter Stone.

In our new paper, we find that LLMs can efficiently do RLHF in-context! Our method, in-context preference learning (ICPL), iterates LLMs writing reward functions, training agents, and putting preferences into context. We see a 30x boost in query efficiency over baseline RLHF!

In multi-object env, why do most Unsupervised Skill Discovery methods fail to learn complex skills like tool use? Because they simply maximize state coverage. Introducing our solution SkiLD: Skill Discovery Guided by Factor Interactions (NeurIPS24) wangzizhao.github.io/SkiLD/

Come join our team at UMass Robotics!! We are hiring at the Associate/Full level for a joint appointment in engineering and computer science. Feel free to reach out if you have any questions. RTs appreciated :) careers.umass.edu/amherst/en-us/…

Save the date! RLDM 2025, The Multi-disciplinary Conference on Reinforcement Learning and Decision Making, is only around the corner. Visit our website to keep an eye on our submission deadlines👀 rldm.org

We're hiring new #nlproc faculty this year! Asst or Assoc Professors in NLP at UMass CICS -- careers.umass.edu/amherst/en-us/…

The call for papers for RLC is now up! Abstract deadline of 2/14, submission deadline of 2/21! Please help us spread the word. rl-conference.cc/callforpapers.…

I'm quite excited about this and still a bit shocked that it works as well as it does. Imitation via distribution matching has always felt like a clunky, brittle way to teach agents. Language + zero-shot RL is natural and scales well, due to the unsupervised nature of RL Zero.

Huge congrats to Prasann Singhal for being one of the 8 CRA Outstanding Undergraduate Researcher Award winners! It has been an absolute privilege to work with Prasann during his time at UT. (And he's applying for PhD programs this year...hint hint...) Prasann's work... 🧵

1.5 yrs ago, we set out to answer a seemingly simple question: what are we *actually* getting out of RL in fine-tuning? I'm thrilled to share a pearl we found on the deepest dive of my PhD: the value of RL in RLHF seems to come from *generation-verification gaps*. Get ready to🤿!

It’s about time! 🎉🎉🎉🎉🎉🎉🎉 nytimes.com/2025/03/05/tec…

I'm extremely proud of the work that Harshit has done and looking forward to seeing what he does next. Congratulations, Harshit!

Reminder that early registration for RLC closes on the 30th! Please register early to save yourself some money and help us get the word out.

Behavioral Foundation Models (BFMs) trained with RL are secretly more powerful than we think. BFM’s directly output a policy believed to be near-optimal given any reward function. Our new work shows that they can actually do much better:

RLZero will be presented at NeurIPS Conference 2025 . Learn more about the work in the thread below: