Introducing our new work on mechanistic intepretability of LLM cognition🤖🧠: why do Transformer-based LLMs have limited working memory capacity, as measured by N-back tasks? (1/7) openreview.net/pdf?id=dXjQgm9…

More evidence that working memory is not persistent activity. Instead, it is dynamic on/off states with short-term synaptic plasticity. Intermittent rate coding and cue-specific ensembles support working memory nature.com/articles/s4158… #neuroscience

How do LLMs learn to reason from data? Are they ~retrieving the answers from parametric knowledge🦜? In our new preprint, we look at the pretraining data and find evidence against this: Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢 🧵⬇️

This looks like a very useful and important contribution!

It would be great to have a precise enough formulation of ‘approximate retrieval’ for this hypothesis to be rigorously tested. There is a concern that virtually any task can be characterized in this way, by appealing to a vague notion of similarity with other tasks.

Open Post-Training recipes! Some of my personal highlights: 💡 We significantly scaled up our preference data! (using more than 330k preference pairs for our 70b model!) 💡 We used RL with Verifiable Rewards to improve targeted skills like math and precise instruction following

In this new preprint @smfleming and I present a theory of the functions and evolution of conscious vision. This is a big project: osf.io/preprints/psya…. We'd love to get your comments!

🚨 New paper at NeurIPS Conference w/ Michael Lepori! Most work on interpreting vision models focuses on concrete visual features (edges, objects). But how do models represent abstract visual relations between objects? We adapt NLP interpretability techniques for ViTs to find out! 🔍

Even ducklings🐣can represent abstract visual relations. Can your favorite ViT? In our new NeurIPS Conference paper, we use mechanistic interpretability to find out!

New paper with Alexander Fung, Pramod RT/ಪ್ರಮೋದ್ ರಾ ತಾ , Jessica Chomik , Nancy Kanwisher @[email protected], Ev (like in 'evidence', not Eve) Fedorenko 🇺🇦 on the representations that underlie our intuitive physical reasoning about the world. Thread 🧵about our new preprint 📄✨linked here: tinyurl.com/intphyslang 1/10

Given a high-quality verifier, language model accuracy can be improved by scaling inference-time compute (e.g., w/ repeated sampling). When can we expect similar gains without an external verifier? New paper: Self-Improvement in Language Models: The Sharpening Mechanism

Today OpenAI announced o3, its next-gen reasoning model. We've worked with OpenAI to test it on ARC-AGI, and we believe it represents a significant breakthrough in getting AI to adapt to novel tasks. It scores 75.7% on the semi-private eval in low-compute mode (for $20 per task

Truly incredible results. I have been impressed with o1’s capabilities but certainly didn’t expect this leap.

Why do pre-o3 LLMs struggle with generalization tasks like ARC Prize? It's not what you might think. OpenAI o3 shattered the ARC-AGI benchmark. But the hardest puzzles didn’t stump it because of reasoning, and this has implications for the benchmark as a whole. Analysis below🧵

New work led by Aaditya Singh: "Strategy coopetition explains the emergence and transience of in-context learning in transformers." We find some surprising things!! E.g. that circuits can simultaneously compete AND cooperate ("coopetition") 😯 🧵👇

🚨 Discover the Science of LLM! We uncover how LLMs (Llama3-70B) achieve abstract reasoning through emergent symbolic mechanisms: 1️⃣ Symbol Abstraction Heads: Early layers convert input tokens into abstract variables based on their relationships. 2️⃣ Symbolic Induction Heads:

Led by postdoc Doyeon Lee and grad student Joseph Pruitt, our lab has a new Perspectives piece in PNAS Nexus: "Metacognitive sensitivity: The key to calibrating trust and optimal decision-making with AI" academic.oup.com/pnasnexus/arti… With co-authors Tianyu Zhou and Eric Du 1/

The final version of this paper has now been published in open access in the Journal of Memory and Language (link below). This was a long-running but very rewarding project. Here are a few thoughts on our methodology and main findings. 1/9

🤖🧠 NEW PAPER ON COGSCI & AI 🧠🤖 Recent neural networks capture properties long thought to require symbols: compositionality, productivity, rapid learning So what role should symbols play in theories of the mind? For our answer...read on! Paper: arxiv.org/abs/2508.05776 1/n