My new paper "Deep Learning is Not So Mysterious or Different": arxiv.org/abs/2503.02113. Generalization behaviours in deep learning can be intuitively understood through a notion of soft inductive biases, and formally characterized with countable hypothesis bounds! 1/12

What if an LLM could update its own weights? Meet SEAL🦭: a framework where LLMs generate their own training data (self-edits) to update their weights in response to new inputs. Self-editing is learned via RL, using the updated model’s downstream performance as reward.

Gemini 2.5 tech report was released!

These mortgages are solely collateralized with fartcoin and SXP6900?

good trajectory++ ⭐️🍿

Reagan:

This will probs be one of the most influential papers of the year

Excited to share what I worked on during my time at Meta. - We introduce a Triton-accelerated Transformer with *2-simplicial attention*—a tri-linear generalization of dot-product attention - We show how to adapt RoPE to tri-linear forms - We show 2-simplicial attention scales

Got a chance to measure Maximum Achievable Matmul TFLOPS on NVIDIA B200. With each new NVIDIA generation the efficiency keeps on dropping: A100: 86.9% H100: 80.3% B200: 77.6% The updated table is here: github.com/stas00/ml-engi…

🚨 BREAKING: Detailed list of all 44 people in Meta's Superintelligence team. — 50% from China — 75% have PhDs, 70% Researchers — 40% from OpenAI, 20% DeepMind, 15% Scale — 20% L8+ level — 75% 1st gen immigrants Each of these people are likely getting paid $10-$100M/yr.

This picture

Beautiful Google Research paper. LLMs can learn in context from examples in the prompt, can pick up new patterns while answering, yet their stored weights never change. That behavior looks impossible if learning always means gradient descent. The mechanisms through which this

This was a very good read yesterday, they fix not only GRPO, but most RL on sequences. They found a bug in a common definition of importance sampling for sequence models and fix just that in a very simple way. No hack required, just correct math!

Excited to share Flow Matching Policy Gradients: expressive RL policies trained from rewards using flow matching. It’s an easy, drop-in replacement for Gaussian PPO on control tasks.

Today we're putting out an update to the JAX TPU book, this time on GPUs. How do GPUs work, especially compared to TPUs? How are they networked? And how does this affect LLM training? 1/n

The older I get the more I realize that in most countries these days governments are more like a mafia that just extorts more and more money from the people Before I saw the government as a benevolent group of people that represented the people, and in return for tax payments

alexander wang with yann lecun

attention sinks may be a bias in causal transformers. as some of you know, i've been writing a long blogpost on attention and its properties as a message-passing operation on graphs. while doing so, i figured i might have found an explanation for which attention sinks may be an

This one's mine. Dante bookcase on the third wall not pictured.

New paper 📜: Tiny Recursion Model (TRM) is a recursive reasoning approach with a tiny 7M parameters neural network that obtains 45% on ARC-AGI-1 and 8% on ARC-AGI-2, beating most LLMs. Blog: alexiajm.github.io/2025/09/29/tin… Code: github.com/SamsungSAILMon… Paper: arxiv.org/abs/2510.04871