🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Wow congratulations Steph!! 🎉✨

Very nice analyses on the benefit & limitations of the parallel mode of generative modeling!!💡

The application is now open for our #KempnerInstitute Research #Fellowship! Postdocs studying the foundations of #intelligence or applications of #AI are encouraged to apply. Learn more and apply by Oct. 1: bit.ly/4djPSf8 #LLMs #NeuroAI #ML Sham Kakade Bernardo Sabatini

CFP & join us at M3L Workshop @ NeurIPS 2024 at #NeurIPS2024! We look forward to learning about your insights & findings on the theoretical and scientific understanding of ML phenomena💡

We've extended the #M3L submission deadline to October 1st AoE to align with ICLR timelines. We look forward to your work!

Chatbots are often augmented w/ new facts by context from the user or retriever. Models must adapt instead of hallucinating outdated facts. In this work w/Sachin Goyal, Zico Kolter, Aditi Raghunathan, we show that instruction tuning fails to reliably improve this behavior! [1/n]

Thank you everyone for the fun workshop & see you next year~! ✨

Our work has been selected as an Oral at ICLR 25! We find theoretical and empirical explanations for the benefits of progressive distillation. Amazing work led by Abhishek Panigrahi and Bingbin Liu, done in collaboration with Andrej Risteski and Surbhi Goel :)

1.5 yrs ago, we set out to answer a seemingly simple question: what are we *actually* getting out of RL in fine-tuning? I'm thrilled to share a pearl we found on the deepest dive of my PhD: the value of RL in RLHF seems to come from *generation-verification gaps*. Get ready to🤿!

1⃣ Distillation (Oral; led by Abhishek Panigrahi and Bingbin Liu): theory + exps on when and how progressive distillation (training w a few intermediate teachers) is beneficial for the student. Prog distillation induces a provably beneficial implicit curriculum. arxiv.org/abs/2410.05464

What is the role of memory for modeling time dependent PDEs? I will be at ICLR presenting our paper (Oral) where we study when it is beneficial for modeling time-dependent PDEs! 🔗openreview.net/forum?id=o9kqa… [Oral]: Thu 24 Apr 10:30 am @ Session 1E [Poster]: Thu 24 Apr 3 pm #617

If you're at #ICLR2025, come to our oral talk & poster on progressive distillation presented by the amazing Abhishek Panigrahi! ✨🌴 Joint work with (the equally amazing) Sadhika Malladi, Andrej Risteski, Surbhi Goel More details at our blog: unprovenalgos.github.io/progressive-di…

New in the Deeper Learning blog: Kempner researchers show how VLMs speak the same semantic language across images and text. bit.ly/KempnerVLM by Isabel Papadimitriou, Chloe H. Su, Thomas Fel, Stephanie Gil, and Sham Kakade #AI #ML #VLMs #SAEs

Misha Khodak, Tanya Marwah, along with myself, Nicholas Boffi and Jianfeng Lu are organizing a COLT 2025 workshop on the Theory of AI for Scientific Computing, to be held on the first day of the conference (June 30).

Excited to announce MOSS, our ICML workshop focused on discoveries at small scale! We believe there's tremendous potential & creativity in research done with limited resources and would love to hear your ideas. The submission (due May 22nd) can literally be a Jupyter notebook! :)

📢 (1/16) Introducing PaTH 🛣️ — a RoPE-free contextualized position encoding scheme, built for stronger state tracking, better extrapolation, and hardware-efficient training. PaTH outperforms RoPE across short and long language modeling benchmarks arxiv.org/abs/2505.16381

How do architectural limitations of Transformers manifest after pretraining?

[1/n] New work [JSKZ25] w/ Jikai Jin, Vasilis Syrgkanis, Sham Kakade. We introduce new formulations and tools for evaluating language model capabilities, which help explain recent observations of post-training behaviors of Qwen-series models — there is a sensitive causal link