🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Vision transformers have high-norm outliers that hurt performance and distort attention. While prior work removed them by retraining with “register” tokens, we find the mechanism behind outliers and make registers at ✨test-time✨—giving clean features and better performance! 🧵

One of the most interesting papers to read at the moment with extreme implications also for language transformers.

We are starting the afternoon session soon! Grand Ballroom C1 on the fourth floor!

How do LMs track what humans believe? In our new work, we show they use a pointer-like mechanism we call lookback. Super proud of this work by Nikhil Prakash and team! This is the most intricate piece of LM reverse engineering I’ve seen!

“How will my model behave if I change the training data?” Recent(-ish) work w/ Logan Engstrom: we nearly *perfectly* predict ML model behavior as a function of training data, saturating benchmarks for this problem (called “data attribution”).

Can diffusion models appear to be learning, when they’re actually just memorizing the training data? We show and investigate this phenomenon in the context of neural network weight generation, in our recent paper “Generative Modeling of Weights: Generalization or Memorization?"

We’re proud to announce three new tenure-track assistant professors joining TTIC in Fall 2026: Yossi Gandelsman (Yossi Gandelsman), Will Merrill (William Merrill), and Nick Tomlin (Nicholas Tomlin). Meet them here: buff.ly/JH1DFtT

The vision mechanistic interpretability workshop Mechanistic Interpretability for Vision @ CVPR2025 earlier this month at CVPR was very informative and fun! Looking forward to seeing this community grow. Thank you to the speakers and organizers trevordarrell David Bau Tamar Rott Shaham Yossi Gandelsman Joanna

Thank you very much to our wonderful speakers and attendees Mechanistic Interpretability for Vision @ CVPR2025 who made the workshop a huge success. We hope to see you again next year! The workshop recording link can be accessed at: youtu.be/LTh86RMAWsI?si….

In a recent paper, physicists used two predictable factors to reproduce the “creativity” seen from image-generating AI. Webb Wright reports: quantamagazine.org/researchers-un…

Check out the updates to our paper. There are other ways to play around with these intriguing "register neurons." Also, check out our updated set of models with test-time registers. We have VLMs too! huggingface.co/collections/am…