Talking to many junior faculty members and students in AI lately. Many seem to be somewhat lost with all the seemingly fast progresses made by the industry. My suggestion to them is: It is industry's job to find how to do better, but academia is to find out how to do it right.

Deciphering a neural network’s insides has been near impossible, and researchers hunt for any clue they can find.

Recently, they discovered a new one.

Anil Ananthaswamy reports:
quantamagazine.org/how-do-machine…

I first heard Irina Rish mention Grokking #neural networks on the Paul Middlebrooks's Brain Inspired podcast! That was in early '22. Years later, here's a story on Grokking, about the follow-up detective work of Neel Nanda Ziming Liu and others for Quanta Magazine quantamagazine.org/how-do-machine…

I quit PhD (for a day) and opened a boba shop at Massachusetts Institute of Technology (MIT) - Generative Boba! It’s a huge success - right next to our office so all the AI researchers are enjoying it. Checkout our boba diffusion algorithm in the poster to understand why boba generation is so important to Massachusetts Institute of Technology (MIT)_CSAIL !

Giving the Presidential Lecture tomorrow at Simons Foundation Flatiron Institute:

'The Next Great Scientific Theory is Hiding Inside a Neural Network' simonsfoundation.org/event/the-next…

Will be in NYC until the 10th – please get in touch if you would like to chat!

🥳🥳🥳 We are excited to share that AI for Science workshop will be held again with ICML Conference 2024, Vienna! This time, we focus on scaling in AI for Science (as a new dimension to theory, methodology and discovery)! Tentative schedules can be found: ai4sciencecommunity.github.io/icml24.html

I'm excited that a Dutch research team has taken our surprising AI-discovered ozone-chemistry conservation law, figured out what it means, and generalized it! I see great potential for AI helping science.😀

A strange conservation law discovered by our AI method is now understood by domain experts (see this paper arxiv.org/abs/2306.09553)! This is a special moment for me - never felt this proud of the tools we're building! And, NO, AI isn't replacing scientists, but complementing us.

We propose Q-probe, a simple technique that improves coding and alignment for LLM, without requiring fine-tuning!. The idea is to learn a 'task vector' in the hidden space and use it to select from multiple candidate generations. arxiv.org/abs/2402.14688

This paper might be bad news for mech interp work that hopes to uncover interpretable programmatic systematicity within transformers. The fact that DeepMind's best attempt to get length generalization for addition works up to but not after length increases of 2.5x demonstrates

Our new paper shows how machine-learning to generalize can be modeled as representations interacting like particles ('repons'). We also predict how much data is needed, and find a Goldilocks zone where the decoder is neither too weak nor too powerful: Ziming Liu David D. Baek