Five minutes of thinking can save you millions of GPU hours.

Ah

🤔 Can simple string-matching metrics like BLEU rival reward models for LLM alignment? 🔍 We show that given access to a reference, BLEU can match reward models in human preference agreement, and even train LLMs competitively with them using GRPO. 🫐 Introducing BLEUBERI:

Accepted to Findings of #EMNLP2025!

Most AI detection tools assume text is either fully human or fully AI. But what about the huge gray zone of AI edits to human text? We tackle this with EditLens, a model that quantifies the magnitude of AI edits to text. Coming soon to Pangram Labs! 1/

🚨New paper on AI and copyright Several authors have sued LLM companies for allegedly using their books without permission for model training. 👩‍⚖️Courts, however, require empirical evidence of harm (e.g., market dilution). Our new pre-registered study addresses exactly this

AI is already at work in American newsrooms. We examine 186k articles published this summer and find that ~9% are either fully or partially AI-generated, usually without readers having any idea. Here's what we learned about how AI is influencing local and national journalism:

Excited to be at EMNLP ‘25 presenting VeriFastScore. Come find me at Hall C (poster 173) between 1pm and 2pm today.

I'm at #NeurIPS2025 from 12/2-12/8! will present 🫐 BLEUBERI at the Wednesday (12/3) 4:30PM poster session w/ Yekyung Kim , come say hi! ☕️ also open to chatting about: • metric dev for measuring long-form outputs • synthetic evals • reasoning in domains outside math/code

Our 🫐BLEUBERI isn't just cute. It's easy and effective for instruction following! Come check out our poster! I'm currently looking for a internship job 👀 Would love to have ☕️ chat about - Long-context eval / training - Narrative reasoning - Factuality in complex claim