🚀 News! Our 2nd Workshop on Long-Context Foundation Models (LCFM), to be held at ICML 2025 in Vancouver 🇨🇦! If you're working on long-context models, consider submitting your work! 🗓️ DDL: May 22, 2025 (AOE) 🌐 Web: longcontextfm.github.io 🔗 OpenReview: bit.ly/lcfmworkshop

LMs often output answers that sound right but aren’t supported by input context. This is intrinsic hallucination: the generation of plausible, but unsupported content. We propose Precise Information Control (PIC): a task requiring LMs to ground only on given verifiable claims.

🤔 Recent mech interp work showed that retrieval heads can explain some long-context behavior. But can we use this insight for retrieval? 📣 Introducing QRHeads (query-focused retrieval heads) that enhance retrieval Main contributions: 🔍 Better head detection: we find a

💡 Curious about long-context foundation models (LFCM)? 🧠 We’re hosting a panel at the LCFM workshop at #ICML2025 on “How to evaluate long-context foundation models?” — We’d love to feature your question! Anything on long-context evaluation or modeling — drop it below / DM me🎤

Excited to share that I will be starting as an Assistant Professor in CSE at UCSD (UCSD CSE) in Fall 2026! I am currently recruiting PhD students who want to bridge theory and practice in deep learning - see here: cs.princeton.edu/~smalladi/recr…

Congrats!!! As an empiricist, I always found your work super relevant and provide useful theoretical insights! Also thanks for giving great advices in our chats :)

I am going to present two papers at #COLM2025 tomorrow from 4:30-6:30pm, as none of our leading authors can attend due to visa issues. Haven't done poster presentations for years 🤣🤣 .... so I will do my best! #76: LongProc #80: Goedel-Prover v1

We will present QRHead (Wuwei Zhang) #EMNLP2025 Without any training, we boosts Llama-3.1-8B’s performance by >10% 📈on context reasoning tasks (CLIPPER, LongMemEval), and outperforms specialized re-rankers on BEIR. Check out our (virtual) poster tonight!