New paper with Leon and Erin Grant! Why do we see localized receptive fields so often, even in models without sparisity regularization? We present a theory in the minimal setting from Alessandro Ingrosso and Sebastian Goldt

2025 @Blavatnikawards UK 🇬🇧 Finalist Andrew Saxe from UCL was featured on the BBC Science Focus Instant Genius Podcast on how AI is helping us understand how our brains🧠 learn: open.spotify.com/episode/6pcv55… and podcasts.apple.com/gb/podcast/how… Andrew Saxe @UCL

Transformers employ different strategies through training to minimize loss, but how do these tradeoff and why? Excited to share our newest work, where we show remarkably rich competitive and cooperative interactions (termed "coopetition") as a transformer learns. Read on 🔎⏬

Our paper, “A Theory of Initialization’s Impact on Specialization,” has been accepted to ICLR 2025! openreview.net/forum?id=RQz7s… We shows how neural network can build specialized and shared representation depending on initialization, this has consequences in continual learning. (1/8)

Happy to share the recording of my plenary talk at Cosyne 2025 two days ago. You will learn about the statistical physics approach, phase transitions in learning, transformers, sequence models, attention etc. youtube.com/watch?v=PurZcs…

How do cognitive maps fail? And how can this help us understand/treat psychosis? My lab at Experimental Psychology, Oxford is hiring a Postdoc tinyurl.com/2p935hhz and RA tinyurl.com/3myfpb78 to answer these questions in mouse models. Here's why you might want to join: 🧵

Our paper, “Make Haste Slowly: A Theory of Emergent Structured Mixed Selectivity in Feature Learning ReLU Networks” will be presented at ICLR 2025 this week (openreview.net/forum?id=27SSn…)! We derive closed-form dynamics for some, remarkably linear, feature learning ReLU networks (1/9)

If I had known about this master when I was coming out of my Bachelor, I would have applied in a heartbeat, so please help me spread the word among the budding physicists with an interest in complex systems !

Our paper just came out in PRX! Congrats to Nishil Patel and the rest of the team. TL;DR : We analyse neural network learning through the lens of statistical physics, revealing distinct scaling regimes with sharp transitions. 🔗 journals.aps.org/prx/abstract/1…

Really happy to see this paper out, led by Nishil Patel in collaboration with Stefano Sarao Mannelli and Andrew Saxe: we apply the statistical physics toolbox to analyse a simple model of reinforcement learning, and find some cool effects, like a speed-accuracy trade-off for generalisation 🚀

Was super fun to be a part of this work! Felt very satisfying to bring the theory work on ICL with linear attention a bit closer to practice (with multi-headed low rank attention), and of course, add a focus on dynamics. Thread 🧵 with some extra highlights

Excited to share this work has been accepted as an Oral at #icml2025 -- looking forward to seeing everyone in Vancouver, and an extra thanks to my amazing collaborators for making this project so much fun to work on :)

How do task dynamics impact learning in networks with internal dynamics? Excited to share our ICML Oral paper on learning dynamics in linear RNNs! with Clémentine Dominé 🍊 Murray Shanahan Pedro Mediano openreview.net/forum?id=KGOcr…