We introduce PiCSAR (Probabilistic Confidence Selection And Ranking)💡: A simple training-free method for scoring samples based on probabilistic confidence, selecting a reasoning chain with the highest confidence from multiple sampled responses. ✏️PiCSAR is generalisable across

🌀Diversity Aware RL (DARLING)🌀 📝: arxiv.org/abs/2509.02534 - Jointly optimizes for quality & diversity using a learned partition function - Outperforms standard RL in quality AND diversity metrics, e.g. higher pass@1/p@k - Works for both non-verifiable & verifiable tasks 🧵1/5

Variational Uncertainty Decomposition for In-Context Learning. arxiv.org/abs/2509.02327

Are LLMs really so prompt-sensitive? 🤔 🚨 Thrilled to share our EMNLP 2025 main conference paper! Prompt sensitivity has long been seen as a core weakness of LLMs—where tiny wording changes flip benchmark results. Our study finds: much of this effect stems from evaluation

Much of the active learning literature avoids modelling the bias and noise of the underlying experimental system...

This is a great example of what good research looks like. You start with a real problem. You peel it layer by layer to find the root cause. You form a new hypothesis and keep digging. At the end, you have something insightful to share!

Why does horizon length grow exponentially as shown in the METR plot? Our new paper investigates this by isolating the execution capabilities of LLMs. Here's why you shouldn't be fooled by slowing progress on typical short-task benchmarks... 🧵

🚨Excited to introduce a major development in building safer language models: Safety Pretraining! Instead of post-hoc alignment, we take a step back and embed safety directly into pretraining. 🧵(1/n)

You're in a ML Engineer interview at Anthropic, and the interviewer asks: "Your LLM inference is running out of GPU memory with long conversations. How do you fix this?" Here's how you answer:

So You Want to Be an Academic? A couple of years into your PhD, but wondering: "Am I doing this right?" Most of the advice is aimed at graduating students. But there's far less for junior folks who are still finding their academic path. My candid takes: anandbhattad.github.io/blogs/jr_grads…

Teaching a new course Stanford University this quarter on explainable AI, motivated by neuroscience. I have curated a paper list 4 pages long (link in comment). What are your favorite papers on explainable AI/mechanistic interpretability that I am missing? Please comment or DM. thanks!

best paper or blog i've read in a while, highly recommend! John is brilliant and his research sets an example for the rest of us. recently i too have been thinking deeply about how many bits might be learned via one step of RL or SFT.. if you're thinking about this too, lmk!

Does each policy gradient update only contain O(1) bits of information about the model we are trying to train while each SFT update has O(T) where T is the number of tokens in the example? I think the answer is not a clean binary. Let’s examine this in a simple setup. Let’s

Strong motivation for us to get the COLM talks up quickly. youtube.com/@colm_conf

Instruction tuning has a hidden cost: ✅ Better at following instructions ❌ Narrower output distribution ❌ Worse in-context steerability We built 🌈 Spectrum Suite to investigate this and 🌈 Spectrum Tuning as an alternative post-training method —

The coverage principle: How pre-training enables post-training New preprint where we look at the mechanisms through which next-token prediction produces models that succeed at downstream tasks. The answer involves a metric we call the "coverage profile", not cross-entropy.

Junior students who have just started doing research? Check out the (75 and counting) awesome tips! github.com/jbhuang0604/aw…

✨ New course materials: Interpretability of LLMs✨ This semester I'm teaching an active-learning grad course at Tel Aviv University on LLM interpretability, co-developed with my student Daniela Gottesman. We're releasing the materials as we go, so they can serve as a resource for anyone

This new research paper claims to complete million-step LLM tasks with zero errors. Huge for improving reliable long-chain AI reasoning. Worth checking out if you are an AI dev. Current LLMs degrade substantially when executing extended reasoning chains. Error rates compound