LLMs often exhibit poorly calibrated confidence, which undermines users' trust in their outputs. Though methods exist for short-form answers, they don't address long-form responses😕 Discover the solution in our #ICLR2024 paper! 📄 arxiv.org/abs/2310.19208 👀

🌍 How Verifiable Are LM Responses in the Wild? A Three-Way Factuality Benchmark Meet 𝐅𝐚𝐜𝐭𝐁𝐞𝐧𝐜𝐡 – an updatable benchmark for evaluating language models' factuality in real-world scenarios. 🔗 huggingface.co/spaces/launch/… LaunchNLP MichiganAI Computer Science and Engineering at Michigan

Heard of the Alaska-Hawaii merger?🤔Wonder if LLMs know it’s pending government approval before it can happen? They stumble, but we’ve got a fix⚒️! Dive into my #EMNLP2024 work 𝐍𝐚𝐫𝐫𝐚𝐭𝐢𝐯𝐞-𝐨𝐟-𝐓𝐡𝐨𝐮𝐠𝐡𝐭—a special prompting technique to unlock LLMs’ temporal reasoning

Hi everyone 👋, I will be presenting our work at #EMNLP2024 on automatically optimizing feedback generation systems for improved implementation performance, on 12th Nov, 14:00 - 15:30 in the Generation and Summarization oral session. See you there!

Evaluating LLM research agents on scientific discovery lacks objective measures for assessing proposed methods. This paper introduces MLRC-BENCH, a benchmark using Machine Learning conference competitions to objectively evaluate agent novelty and effectiveness against human

🚨Announcing SCALR @ COLM 2025 — Call for Papers!🚨 The 1st Workshop on Test-Time Scaling and Reasoning Models (SCALR) is coming to Conference on Language Modeling in Montreal this October! This is the first workshop dedicated to this growing research area. 🌐 scalr-workshop.github.io

🔍LLMs now give medical diagnoses, legal advice, and even tackle scientific problems. ❓Your LLM sounds smart. But what if it’s just good at faking expertise? 🚀We built ExpertLongBench to find out. 📉And the results? They revealed several concerns.👇 🔗 huggingface.co/spaces/launch/…

🚨 Deadline for SCALR 2025 Workshop: Test‑time Scaling & Reasoning Models at COLM '25 Conference on Language Modeling is approaching!🚨 scalr-workshop.github.io 🧩 Call for short papers (4 pages, non‑archival) now open on OpenReview! Submit by June 23, 2025; notifications out July 24. Topics

🔥 Excited to introduce ManyICLBench (ACL 2025) 🧐 Do many-shot ICL tasks evaluate LCLMs' ability to retrieve the most similar examples or learn from many examples? We carefully analyzed numerous tasks and categorized them. 📄 Paper: arxiv.org/abs/2411.07130 #ACL2025