🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

I’m excited to share that our paper has been accepted at ICML 2025! 🎉🥳🎊 This work was done during my internship at IBM Research, and it wouldn’t have been possible without a top-notch team Muneeza Azmat Rå¥å Mikhail Yurochkin and my amazing advisor Jessy Li 👏

To appear #ICML2025 🎉 Congrats to Hongli!

Welcome Elias Stengel-Eskin!!! Super excited that you’ll be joining us 🥳

And welcome Chan Young Park !!! 🎊🙌🍾

I’ve been fascinated lately by the question: what kinds of capabilities might base LLMs lose when they are aligned? i.e. where can alignment make models WORSE? I’ve been looking into this with Christopher Potts and here's one piece of the answer: randomness and creativity

🆕paper: LLMs Get Lost in Multi-Turn Conversation In real life, people don’t speak in perfect prompts. So we simulate multi-turn conversations — less lab-like, more like real use. We find that LLMs get lost in conversation. 👀What does that mean? 🧵1/N 📄arxiv.org/abs/2505.06120

✨ Our paper #PlanGenLLMs: A Modern Survey of LLM Planning Capabilities (arxiv.org/pdf/2502.11221) is accepted to the #ACL2025 main conference! Huge thanks to the reviewers for the unanimous 4-4-4 reviews and meta score ❤️ Grateful for your thoughtful feedback! #ACL2025 #NLProc

Introducing ChartMuseum🖼️, testing visual reasoning with diverse real-world charts! ✍🏻Entirely human-written questions by 13 CS researchers 👀Emphasis on visual reasoning – hard to be verbalized via text CoTs 📉Humans reach 93% but 63% from Gemini-2.5-Pro & 38% from Qwen2.5-72B

Yayyyyyy welcome 🤘🤘

Super thrilled that Kanishka Misra 🌊 is going to join UT Linguistics Dept as our newest computational linguistics faculty member -- looking forward to doing great research together! 🧑‍🎓Students: Kanishka is a GREAT mentor -- apply to be his PhD student in the upcoming cycle!!

How good are LLMs at 🔭 scientific computing and visualization 🔭? AstroVisBench tests how well LLMs implement scientific workflows in astronomy and visualize results. SOTA models like Gemini 2.5 Pro & Claude 4 Opus only match ground truth scientific utility 16% of the time. 🧵

Is AI ready to play a real role in science? This work with CosmicAI evaluates LLMs targeting the implementation of scientific workflows, and the scientific utility of visualizations from LLM-generated code -- and the answer is not yet, even with the best SOTA models 👇

Language is often strategic, but LLMs tend to play nice. How strategic are they really? Probing into that is key for future safety alignment.🛟 👉Introducing CoBRA🐍, a framework that assesses strategic language. Work with my amazing advisors Jessy Li and David Beaver! 🧵👇

CosmicAI collab: benchmarking the utility of LLMs in astronomy coding workflows & focusing on the key research capability of scientific visualization. Sebastian Joseph Jessy Li Murtaza Husain Greg Durrett Dr. Stephanie Juneau paul.torrey Adam Bolton, Stella Offner, Juan Frias, Niall Gaffney

We have very good frameworks for cooperative dialog… but how about the opposite? Asher Zheng’s new paper takes a game-theoretic view and develops new metrics to quantify non-cooperative language ♟️ Turns out LLMs don’t have the pragmatic capabilities to perceive these…

Check out this new opinion piece from Sebastian and Lily! We have really powerful AI systems now, so what’s the bottleneck preventing the wider adoption of fact checking systems, in high stakes scenarios like medicine? It’s how we define the tasks 👇

If you’ll be at #icml2025, check out Hongli’s work on context-specific principles!

Excited to share that QUDsim has been accepted to #COLM2025!! 🎉🎉

Happy to share that EvalAgent has been accepted to #COLM2025 Conference on Language Modeling 🎉🇨🇦 We introduce a framework to identify implicit and diverse evaluation criteria for various open-ended tasks! 📜 arxiv.org/pdf/2504.15219

👇Happening this afternoon 4:30pm! Come meet Mikhail Yurochkin, Rå¥å, and I, at East Exhibition Hall #1103. 📍I’m also on the industry job market this coming year! Let’s connect and chat about opportunities in the industry :)