I'm starting a new project. Working on what I consider to be the most important problem: building thinking machines that adapt and continuously learn. We have incredibly talent dense founding team + are hiring for engineering, ops, design. Join us: adaptionlabs.ai

Mathieu is great!

Mila's annual supervision request process is now open to receive MSc and PhD applications for Fall 2026 admission! For more information, visit mila.quebec/en/prospective…

Nadia is a pleasure to work with - really recommend! 🚀

The Maximal Update Parameterization (µP) allows LR transfer from small to large models, saving costly tuning. But why is independent weight decay (IWD) essential for it to work? We find µP stabilizes early training (like an LR warmup), but IWD takes over in the long term! 🧵

🚨 We’re recruiting new students for Fall 2026 — come join Chandar Lab! 🚨

A talk on our recent paper on zero-sum learning, now with new links to generalization, circuit emergence, and optimizer dynamics! 🚀

LLMs memorize a lot of training data, but memorization is poorly understood. Where does it live inside models? How is it stored? How much is it involved in different tasks? Jack Merullo & Srihita Vatsavaya's new paper examines all of these questions using loss curvature! (1/7)

Really cool finding (and paper), and makes a ton of sense! Way back in the day when we/others were trying to make sense of memorization vs. generalization in simpler models, we made an argument that generalizing signals will be top eigenvalues due to shared structure, while 1/2

Ready for day 3 of #EMNLP2025 🎉🎉 I've been on the lookout for memorization, unlearning, interp, memory module papers & more, chat w me if these topics fascinate you too😻 Looking forward to more of Suzhou, the conf & my BlackboxNLP keynote Sunday 1.45PM! blackboxnlp.github.io/2025/

New research: are prompting and activation steering just two sides of the same coin? Eric Bigelow Daniel Wurgaft Ekdeep Singh and coauthors argue they are: ICL and steering have formally equivalent effects. (1/4)

Very cool analysis of pruning vs merging experts in SMoE models! 🔥

Reasoning models are powerful, but they burn thousands of tokens on potentially wrong interpretations for ambiguous requests! 👉 We teach models to think about intent first and provide all interpretations and answers in a single response via RL with dual reward. 🧵1/6

1/ We found that deep sequence models memorize atomic facts "geometrically" -- not as an associative lookup table as often imagined. This opens up practical questions on reasoning/memory/discovery, and also poses a theoretical "memorization puzzle."

Recent paper from our lab on Can LLMs become CAD designers? 🛠️

Recent paper from our lab on LLMs playing Hangman 🎣

Excited to share that we have 3 papers accepted at #ICLR2026! 🇧🇷 Our work this year focuses on efficiency and expressivity: deriving theoretical limits for SSMs, achieving linear scaling for reasoning, and modernizing encoder architectures. A summary of our work 👇 🧵

🚨 New Paper Alert: Why LLM training follows a slow power law? ⁉️We find the neural scaling law with time arises intrinsically from softmax and cross-entropy! (1/6) When learning peaked (or low-temperature or low-entropy) distributions like next-token distributions, these