🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Induction heads are commonly associated with in-context learning, but are they the primary driver of ICL at scale? We find that recently discovered "function vector" heads, which encode the ICL task, are the actual primary drivers of few-shot ICL. arxiv.org/abs/2502.14010 🧵

Past work has shown that world state is linearly decodable from LMs trained on text and games like Othello. But how do LMs *compute* these states? We investigate state tracking using permutation composition as a model problem, and discover interpretable, controllable procedures🧵

Join us this week at the #NAACL student research workshop for an exciting keynote presentation by Philip Resnik! Fun fact, he was a co-creator of the SRW back in the day, so this is coming back full circle ⭐ ⏱️ May 1st, 4-5PM MDT 📍 San Miguel, or online

Returning from #NAACL2025 & had some interesting discussions! One topic that came up a lot was AI scientists and how they should be implemented, evaluated, etc. Used this as inspiration to finish up a blog post on AI science and the state of reviewing: nightingal3.github.io/blog/2025/04/2…

I do think that AI has a lot of promise for science, but we need lots of serious work to get there! One thing I'm very interested in is how we can build AI systems that can effectively judge the quality of research.