We are still in an Evaluation crisis!

Ahmad Beirami ✈️ NeurIPS Indeed! In our NeurIPS workshop LLM Evals Workshop @NeurIPS, we’ll tackle the most pressing evaluation challenges. Join us to discuss how we should design the next generation of evaluations with experts in the field! More details: sites.google.com/view/llm-eval-…

Deadline for LLM Evals Workshop @NeurIPS submissions is September 4th! If you’re interested in helping us as a reviewer, please volunteer 👇

Volunteer to be a reviewer: docs.google.com/forms/d/1JPQB2…

Last 2 days to the deadline! If you're interested in helping with the reviewing process, please consider volunteering using the following form. There will be best reviewer awards! docs.google.com/forms/d/e/1FAI…

The Narcissus Hypothesis: --Recursive training on semi-synthetic corpora enforcing human alignment induces a Social Desirability Bias: world-models (Narcissus) aim to please rather than represent, polluting data lakes and charming us (Echo) into hanging on their every word.

Sketched on a few Parisian summer nights with a friend, Christian Internò . If you care about (causal) identification in a semi-synthetic future, we’d value your read and critique. Preprint: arxiv.org/pdf/2509.17999 Accepted at LLM Evals Workshop @NeurIPS workshop NeurIPS Conference

I’ll be NeurIPS Conference all week and would love to connect on LLM data, evaluation, benchmarking, and scaling laws. If you’re working on related problems, feel free to reach out. PS: Don’t miss our one-of-a-kind workshop on LLM evaluation: sites.google.com/view/llm-eval-…