🧠🤖 Does learning in the brain inherently require plasticity? In our latest paper we question this assumption, by leveraging insights into how LLMs "learn". Check out this thread for more details! biorxiv.org/content/10.110… w/ Yuval Shalev Gabriel Stanovsky Ariel Goldstein

Exciting news! I'll present my poster at #ACL2024 about unsupervised document structure extraction tomorrow (Aug. 12th) at 12:45 PM 🕒 Come say hi and let's chat over the paper! arxiv.org/pdf/2402.13906 More details below ⬇️ w/ Gabriel Stanovsky (((ل()(ل() 'yoav))))👾 Ai2 HUJI NLP

📢Paper release📢 What computation is the Transformer performing in the layers after the top-1 becomes fixed (a so called "saturation event")? We show that the next highest-ranked tokens also undergo saturation *in order* of their ranking. Preprint: arxiv.org/abs/2410.20210 1/4

📄 New paper: "A Minimum Description Length Approach to Regularization in Neural Networks" with Orr Well, Emmanuel Chemla, Roni Katzir, and Nur Lan . We explore why neural networks often struggle with simple structured tasks. Spoiler: our regularizers might be the problem. 🧵