🚨New preprint 🚨 I’m super excited to share our work: To Code, or Not To Code? Exploring the Impact of Code in Pre-training 📜: arxiv.org/abs/2408.10914 w/ Yixuan Su, raymond ma, Adrien Morisot, Ivan Zhang, Acyr Locatelli, Marzieh Fadaee, Ahmet Üstün, Sara Hooker [1/n]

Very excited, obviously about the work, but also because I finally got to make a Taylor Swift reference in a paper title!!

These days RAG systems have gotten popular for boosting LLMs—but they're brittle💔. Minor shifts in phrasing (✍️ style, politeness, typos) can wreck the pipeline. Even advanced components don’t fix the issue. Check out this extensive eval by Neel Bhandari and Tianyu (Tiya) Cao!

Real user queries often look different from the clean, concise ones in academic benchmarks - ambiguity, full of typos, and much less readable. We show that even strong RAG systems quickly break under these conditions. Awesome project led by Neel Bhandari and Tianyu (Tiya) Cao!!

Excited to share PolyGuard 🛡️, our new state-of-the-art multilingual safety detector. PolyGuard supports 17 languages and outperforms all open-source and commercial moderation tools!

Very proud of this work which is being presented ICLR 2026 later today. While I will not be there — Catch up with Viraat Aryabumi and Ahmet Üstün who are both fantastic and can share more about our work at both Cohere Labs and cohere. 🔥✨

Excited to announce our #NAACL2025 Oral paper! 🎉✨ We carried out the largest systematic study so far to map the links between upstream choices, intrinsic bias, and downstream zero-shot performance across 131 CLIP Vision-language encoders, 26 datasets, and 55 architectures!

LMArena is widely used for model evaluation, but is it measuring true progress? 🔮 In our work, "The Leaderboard Illusion", we reveal: 🔒 Private testing 📊 Data access asymmetries ⚠️ Overfitting risks 🚫 Silent deprecations Despite best intentions, arena policies favor a few!

Super excited and honored to received this award! 🥰

Excited to teach Advanced NLP at CMU again this semester! Slides are on the course page as the course proceeds: cmu-l3.github.io/anlp-fall2025/ Lectures will be uploaded to Youtube: youtube.com/playlist?list=…