40% with just 1 try per task: SWE-agent-LM-32B is the new #1 open source model on SWE-bench Verified. We built it by synthesizing a ton of agentic training data from 100+ Python repos. Today we’re open-sourcing the toolkit that made it happen: SWE-smith.

Hieu Pham My friend and lab mate Ruiqi Zhong has a great blog post reflecting many PhD students' thoughts on this: ruiqizhong.substack.com/p/is-a-phd-on-…

I think more and more people will realize RL is RL — Deep RL (gradient-based) is a particular solution for RL. Focus on the problem — solve it however you want.

There are no intuitions about what is going on here. MDPs are a bad model for real RL problems, and even for most toy ones. RL is hard to explain because your data comes from interacting with an environment. Non-stationary + hard to make fast!

We're organizing workshops on Programmatic Representation for Agent Learning at the upcoming #ICML2025 and #RLC2025. We welcome contributions using programs as policies, reward functions, skill libraries, task generators, environment models, etc., and more! See you soon!😀

One of our newest pre-training projects was built with Marin! Stay tuned for more soon! Thanks for Elyas Obbad & David Hall for being so fun to work with -- and Percy Liang help test Marin & Sanmi Koyejo really good kind advice. & Rylan Schaeffer for his very efficient feedback ;)

How should we order training examples? In a new blogpost (w/ Yiding Jiang), we explore a compression-based perspective: order your dataset to minimize its prequential codelength.

Nice example of low hanging fruit connecting the dots. Some of the comments with links to papers suggest this is not a denovo discovery & may have been obvious in hindsight. But a lot of things are obvious in hindsight. 1/

Our new preprint is online ! Structural assumptions on the MDP helps in imitation learning, even if offline :) Joint work with Gergely Neu and Antoine Moulin 😎

Your language model is wasting half of its layers to just refine probability distributions rather than doing interesting computations. In our paper, we found that the second half of the layers of the Llama 3 models have minimal effect on future computations. 1/6

I think this type of “tiny”-training is another viable path to make RL work on real world tasks.

Tech is overestimated in the short term, (because infra is so damn harder than people realize) And underestimated in the long run. (becuase new tech becomes infra for new applications) Applies for computer, chip, internet, llm, rl, etc.

Can Large Reasoning Models Self-Train? We propose Self-Rewarded Training (SRT)—where LLMs generate their own supervision. Main findings: SRT initially matches RL on ground truth, but sustained training risks reward hacking. We also investigate mitigation strategies.

✨ New blog post 👀: We have some very fast AI-generated kernels generated with a simple test-time only search. They are performing close to or in some cases even beating the standard expert-optimized production kernels shipped in PyTorch. (1/6) [🔗 link in final post]

I'm experiencing the 🤩 moment when an amazing company just built their new library on top of the framework I helped build! Trace is a library for creating **extremely flexible** LLM-based workflows. Syftr uses Trace to optimize their workflow and push the cost-accuracy Pareto

"Tell, Don't Show" was accepted to #ACL2025 Findings! Our simple approach for literary topic modeling combines the new (language models) with the old (classic LDA) to yield better topics. A possible addition to your CSS/DH research 🛠️ box ✨📚 arxiv.org/abs/2505.23166

I'm onboarding a research dev from France for Trace today with Ching-An and Adith. None of us knew him before. He just shipped, built, and impressed everyone 😅

We've got a major update to our preprint on adversarial regret guarantees for Thompson sampling! As before, I think this is one of the most important projects I've worked on due to new algorithmic primitives that it - in principle - unlocks. Thread below on what's new!

I read this paper in detail, and I am very sad! They literally re-do the optimal reward baseline work that we have known since forever, without even crediting the true authors in their derivations. The third screenshot is taken from: ieeexplore.ieee.org/stamp/stamp.js… As you see, they

Given the sheer number of ppl interested in PG methods nowadays I'm sure innocent "rediscoveries" like this are happening everyday. Otoh, due diligence takes minimal effort today as you can just DeepResearch. All it takes is the sense/taste to ask "no way this is not done b4"...