New on the blog: An interview with former postdoc Wenpeng_Yin, who talks with us about NLP, gardening, and his work at Penn State! ccgblog.seas.upenn.edu/2024/04/interv…

Can GPT-4V and Gemini-Pro perceive the world the way humans do? 🤔 Can they solve the vision tasks that humans can in the blink of an eye? 😉 tldr; NO, they are far worse than us 💁🏻‍♀️ Introducing BLINK👁 zeyofu.github.io/blink/, a novel benchmark that studies visual perception

Our workshop, AI4Research (ai4research.github.io), is looking for reviewers who have expertise in areas such as AI4Material, AI4Medical, AI for Single-Cell, and AI4Education. Please let us know if you are interested.

New on our blog: we’re excited to share our #NAACL2024 papers! ccgblog.seas.upenn.edu/2024/05/naacl-… Congratulations to Sihao Chen, Xiaodong Yu, Chaitanya Malaviya, Ben Zhou, Hangfeng He, Hongming Zhang, Dan Roth, Xingyu Fu, 🌴Muhao Chen🌴, Mark Yatskar, and co-authors! (image Yu et al., 2024)

Excited to share new work done Google DeepMind: 🏔️ DOLOMITES: Domain-Specific Long-Form Methodical Tasks, a new long-form generation benchmark for evaluating language models on **realistic** domain-specific tasks. Website: dolomites-benchmark.github.io Paper: arxiv.org/abs/2405.05938

New on the blog: An interview with former CCG student researcher Celine Lee, who talks with us about code, creativity, and making connections in the NLP community! ccgblog.seas.upenn.edu/2024/06/interv…

I can’t make it to #ACL2024 in person this year, but I’ll be there virtually! 🎉💻 Thrilled to present our three papers on Complex Data Reasoning—Visual Flowcharts QA, Chart Fact-Checking, and Robustness in Finance QA. 🚀📊📚🔍 Stay tuned for details! Cognitive Computation Group UPenn NLP ASU School of Computing and Augmented Intelligence

We’re excited to share our #ACL2024 conference and findings papers in our newest blog post: ccgblog.seas.upenn.edu/2024/08/ccg-pa…. Check out these papers from today’s sessions and next week’s virtual Findings presentations! Congrats to Vivek Gupta Yi Zhang Dan Roth and their co-authors!

Congrats to Peter Baile Chen, Yi Zhang, Dan Roth for the Outstanding Paper Award at #ACL2024 Workshop on Knowledgeable LMs! Is Table Retrieval a Solved Problem? Exploring Join-Aware Multi-Table Retrieval Peter Baile Chen, Yi Zhang, Dan Roth arxiv.org/pdf/2404.09889

With special congratulations to Peter Chen, Yi Zhang, and Dan Roth for their Outstanding Paper Award at the #ACL2024 Workshop on Knowledgeable LMs! "Is Table Retrieval a Solved Problem? Exploring Join-Aware Multi-Table Retrieval" Peter Baile Chen, Yi Zhang, Dan Roth

Before launching into our new semester, we asked our six summer interns to tell us about their experiences working with us this summer. Take a look! Special thanks to Vivek Gupta and Sihao Chen for their excellent mentoring! ccgblog.seas.upenn.edu/2024/08/summer…

Excited to share ✨ Contextualized Evaluations ✨! Benchmarks like Chatbot Arena contain underspecified queries, which can lead to arbitrary eval judgments. What happens if we provide evaluators with context (e.g who's the user, what's their intent) when judging LM outputs? 🧵↓

Grateful for Professor Bill Labov and for all his detailed and far-reaching work in the field of sociolinguistics.

New interview with 🌴Muhao Chen🌴, former CCG postdoc, who talks with us about cats and hamsters, LLM safety, and far-flung national parks! ccgblog.seas.upenn.edu/2025/01/interv…

We’re excited to share our #NAACL2025 conference and findings papers on our blog: ccgblog.seas.upenn.edu/2025/03/ccg-pa…! Congratulations to Sihao Chen Chaitanya Malaviya Siyi Liu Yi Zhang Vivek Gupta Dan Roth and their co-authors! (image Mukhopadhyay et al., 2025)

#ICLR2025 Oral LLMs often struggle with reliable and consistent decisions under uncertainty 😵‍💫 — largely because they can't reliably estimate the probability of each choice. We propose BIRD 🐦, a framework that significantly enhances LLM decision making under uncertainty. BIRD

Excited to share our papers at #ICLR2025 in Singapore! Check out the summaries on our blog (ccgblog.seas.upenn.edu/2025/04/ccg-pa…), and then check out the papers at oral session 1B (BIRD) and poster session 2 (for all three)! Yu Feng, Xingyu Fu, Ben Zhou, 🌴Muhao Chen🌴, Dan Roth

Pleased to share our #NAACL2025 papers in Albuquerque! We’ve updated our blog post with the various session times – come check them out! ccgblog.seas.upenn.edu/2025/03/ccg-pa… Sihao Chen, Chaitanya Malaviya, Siyi Liu, Yi Zhang, Vivek Gupta, Dan Roth

I will be in #ICML2025 next week and present #ReFocus on Tuesday afternoon. 📍 West Exhibition Hall B2-B3 #W-202 ⏱️ Tue 15 Jul 4:30 p.m. PDT - 7 p.m. PDT Happy to chat and connect! Feel free to DM 😁 ReFocus link: huggingface.co/datasets/ReFoc…

Check out our papers at #ICML2025 in Vancouver! Summaries, links, and poster sessions on our blog: ccgblog.seas.upenn.edu/2025/07/ccg-pa…. Congrats to Xingyu Fu, Dan Roth, and their co-authors!