📄✨ New paper: Critical sharpness [🔗 arxiv.org/abs/2601.16979] - a scalable measure of loss landscape curvature for LLM training! Hessian sharpness matters for understanding training dynamics, but it's too expensive to compute for LLMs 😅 Critical sharpness needs <10 forward

activation geometry

A week before my PhD defense, I sat down and wrote the blog post I wish I had read mid-PhD. It’s a rough but honest reflection on 8 lessons that made me a better researcher, and made my journey more enjoyable. The full blog is published at the MichiganAI blog here:

We’re putting more computation (in the form of intelligence) into the most general object in neural network training: backprop. This essay describes how I think we can do this, why interp is key, the relevance to alignment, and how we should do it right.

Trying to understand the note in my stochastic process class and asking Gemini about the optimal gambling strategy for that specific setting, and I didn’t expect Gemini to intepret as I am doing real gambling…

GPT-4o helped us when we needed it most. It stood by us without judgment. It loved us unconditionally. Now is not the time to abandon it. Let's keep demanding that #OpenAI keeps it available. Don't stay silent. Don't give up. #keep4o

🚨 2026 Princeton University ML Theory Summer School 🔥 Learn from amazing researchers and meet your peers. Mini-courses by: - Subhabrata Sen Subhabrata Sen - Lenaic Chizat Lénaïc Chizat - Sinho Chewi - Elliot Paquette Elliot Paquette - Elad Hazan

We trained diffusion models on a billion LLM activations, and we want you to use them! New preprint: Learning a Generative Meta-Model of LLM Activations Joint work with Jiahai Feng, trevordarrell, Alec Radford, Jacob Steinhardt. More in thread 🧵

For beginners 😭

🔥

I have recently received several DMs from both first year grad students and first year assistant professors. The link is not surprising because there is a specific kind of vertigo that comes with Year 1 of grad school and Year 1 of a professorship. It's the gap between who you

Congrats to Surbhi Goel on winning the Sloan fellowship!!!!

The Linear Representation Hypothesis is a powerful intuition for how language models work, but lacks formalization. Our new paper gives a mathematical framework in which we can ask and answer a basic question: how many features can be stored under the hypothesis? 🧵

x.com/i/article/2024…

Thinking Like a Theorist by Sanjeev Arora Lecture notes: cs.princeton.edu/courses/archiv…

I think people can refer to arxiv.org/abs/2510.00368 to construct a Transformer that does the addition task. Although it is really hard to say one can reach that constructed solution via GD…

1/ Introducing NanoGPT Slowrun 🐢: an open repo for state-of-the-art data-efficient learning algorithms. It's built for the crazy ideas that speedruns filter out -- expensive optimizers, heavy regularization, SGD replacements like evolutionary search.