Wrapping up the semester with an amazing party hosted by Becky! 🎉 Missed a few of our NLP crew today

Congratulations to Dr. Zhaohui Li (ZhaohuiLee) for completing his dissertation with flying colors and winning the "Best Doctoral Dissertation" award. His impactful works on AI in education are already having real-world impacts. Wishing you all the best for the future!!🌟

📢 New survey on Self-Correction of LLMs! 😢 LLMs often cannot correct their mistakes by prompting themselves 😢 Many studies conduct unfair experiments 😃 We analyze requirements for self-correction🧵 Yusen Zhang✈️@COLM’24 Nan Zhang Jiawei Han Rui Zhang arxiv.org/abs/2406.01297

We present Chain of Agents, a novel framework that harnesses multi-agent collaboration through natural language to enable information aggregation and context reasoning across various LLMs over long-context tasks! Led by Yusen Zhang✈️@COLM’24 from his internship at Google

📢Training efficient LLMs toward a knowledge-intensive domain? Consider domain-specific and task-agnostic compression! Code: github.com/psunlpgroup/D-… Paper: arxiv.org/abs/2405.06275 Excited to present our #NAACL findings paper for LLMs pruning during Virtual Poster Session 2!🧵

⚡️We present D-Pruner for LLM pruning to create domain-specific, task-agnostic LLMs by jointly identifying LLM weights that are pivotal for general capabilities and domain-specific knowledge! NAACL 2024 Findings led by Nan Zhang

Finally in 5th (maybe 6th?) attempt over past 16 months, my paper has been accepted! It's been a marathon, but now I can peacefully focus on other projects. *sigh of relief*

Our latest work (link: arxiv.org/pdf/2406.16203) is online. We found that LLMs' seemingly super-human performance is due to inappropriate evaluation. For example, when the gold label is unavailable, LLMs still choose from incorrect labels. We define and benchmark this new problem.

🚨There is serious lack of robustness with MMLU! In our new work we find that “Changing Answer Order Can Decrease MMLU Accuracy” and the accuracy of top models can drop by 10-20%📉 This means leaderboards might not be as reliable as we thought! 📄arxiv.org/abs/2406.19470 (1/N)

Checkout this new work which shows lack of robustness in MMLU!

Happy to share that this work got accepted at Conference on Language Modeling! We introduce a new dataset and methodology to have robust and reliable measurement of biases in LMs. Kudos to the organizers for constructive reviews and the rebuttal phase. See you all in Philly!

👏How to reduce the uncertainty of LLM generation? Beyond Self-Consistency, we can ask i) Direct prompt, "Which one is correct?" and ii) Inverse prompt, "Which one is incorrect?". Humans are consistent with them, how about LLMs? joint work w/ Lu Cheng Rui Zhang

🥳🥳I'm thrilled that our work that introduces the state space models (SSMs) into de novo design is now published at Nature Communications 🎉 🎉 nature.com/articles/s4146… w/ Sarah de Ruiter, Emanuele Criscuolo, and Francesca Grisoni Molecular Machine Learning

min 15 puan gelir

Smiling faces at our customary NLP Fall party hosted by Becky!