New Preprint from Yukun Huang! Can an LLM determine when its responses are incorrect? Our latest paper dives into "Calibrating long-form generations from an LLM". Discover more at arxiv.org/abs/2402.06544 (1/n)

🧵When should LLMs trust external contexts in RAG? New paper from Yukun Huang and Sanxing Chen enhances LLMs’ *situated faithfulness* to external contexts -- even when they are wrong!👇

excited to see a strong trend on making RAG more reliable to noisy external contexts. just feel the need to compile a list of sparks spotted on X ✨

📢 New preprint on a benchmark for multimodal information extraction! Structured data extraction from long documents consisting of interconnected data in text, tables, and figures remains a challenge. MatViX aims to fill this gap. matvix-bench.github.io

My lab at Duke has multiple Ph.D. openings! Our mission is to augment human decision-making by advancing the reasoning, comprehension, and autonomy of modern AI systems. I am attending #emnlp2024, happy to chat about PhD applications, LLM agents, evaluation etc etc!

I am recruiting PhD students at Duke! Please apply to Duke CompSci or Duke CBB if you are interested in developing new methods and paradigms for NLP/LLMs in healthcare. For details, see here: monicaagrawal.com/home/research-…. Feel free to retweet!

Ever wonder if spotting fake news is getting harder or easier? 🤔 Turns out, despite knowledge cut-offs, popular PolitiFact fact-checks are actually becoming easier for LLMs over time! No real-time data needed. Is our world just getting more predictable for LLMs? 🌍 Fun audio

Glad to share a new ACL Findings paper from @MaxHolsman and Yukun Huang! We introduce Fuzzy Speculative Decoding (FSD) which extends speculative decoding to allow a tunable exchange of generation quality and inference acceleration. Paper: arxiv.org/abs/2502.20704

Looking forward to visiting Stanford this Thursday! Check out my talk at the NLP seminar if you’re around :)

Citations are crucial for improving the trustworthiness of LLM outputs. But can we train LLMs to cite their pretraining data *without* retrieval? New paper from Yukun Huang and Sanxing Chen @ ACL “CitePretrain: Retrieval-Free Knowledge Attribution for LLMs” arxiv.org/pdf/2506.17585

Thanks for sharing our work! “Source-Aware Training Enables Knowledge Attribution in Language Models” is one of the foundational efforts in this space, and we’ve drawn many valuable insights from it. Excited to make this area continue to grow!

If you are at #ACL2025NLP, come checkout our poster on Hall 4X, board 370, 11AM today. Happy to chat about RAG, LLM tool agent, RL for sequential decision making, and everything else! arxiv.org/abs/2410.14651