Excited to announce v0.5 of the Google AI Learning Interpretability Tool (🔥LIT), an interactive platform to debug, validate, and understand ML model behavior. v0.5 includes exciting features and a new name! pair-code.github.io/lit/ #NLProc #googlePAIR (1/7)

David Bau giving keynote 3 on Direct Model Editing and Mechanistic Interpretability

People have been asking for slides of our ACL 2020 tutorial w/ Sebastian Gehrmann Ellie Pavlick Brown NLP on Interpretability and analysis of #nlproc. Thanks to ACL Anthology team it’s now here: aclanthology.org/2020.acl-tutor… Hopefully still useful though much has changed in the field since.

🚨🆕📄🚨 How gold is your human evaluation? We seek the answer, and its implications in the GPT3 era, in our preprint “Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation” Paper: arxiv.org/abs/2212.07981 Equal contribution Yixin Liu

You can explore the ACU annotations in Rose🌹along with protocol results on our demo page and start using our dataset! Repo: github.com/Yale-LILY/ROSE Demo page: yale-lily.github.io/ROSE/ Dataset: huggingface.co/datasets/Sales…

Human preference != Job Done. Check our interesting findings (ROSE🌹) on summarization! Thanks to my fantastic collaborators Alex Fabbri Yixin Liu Pengfei Liu Yilun Zhao Linyong Nan Ruilin Han Sophia Simeng Han Shafiq Joty Caiming Xiong Dragomir Radev!

Very excited to have the opportunity to present research done at Salesforce AI Research on Automatic Text Summarization at Zespół Inżynierii Lingwistycznej IPI PAN „Long Story Short: A Talk about Text Summarization” will cover the current state of the field, existing challenges, and future directions.

How can NLP help us understand the diversity of news coverage of a topic? Check out the latest work from Philippe Laban et al. appearing at #CHI2023 this week.

Delighted to announce our paper has been accepted for an oral presentation at #ACL2023 oral! In this work we emphasize the intricate complexity of human evaluation while it is becoming even more crucial for both model training and evaluation in the LLM era.

Finding a document too dense to decipher? 🤔Content a bit convoluted? Essay too esoteric? Check how we simplify and improve document readability using SWiPE. Join us in making knowledge accessible to all! 🌐 🔗Paper: arxiv.org/abs/2305.19204 🔗Github: github.com/salesforce/sim…

By aligning Wikipedia articles to their simplified versions on Simple Wikipedia, we reconstruct the process by which human editors simplify whole documents, in contrast to prior work focused on sentence-level simplification.

Check out our work on text simplification! SWiPE accepted at #ACL2023

"SWIPE: A Dataset for Document-Level Simplification of Wikipedia Pages" leveraging the entire revision history when pairing enwiki/simplewiki pages, to identify simplification edits. (Laban et al, 2023) arxiv.org/pdf/2305.19204… Wojciech Kryściński

Check out our latest work on text simplification at #acl2023nlp .

🤔Which words in your prompt are most helpful to language models? In our #ACL2023NLP paper, we explore which parts of task instructions are most important for model performance. 🔗 arxiv.org/abs/2306.01150 Code: github.com/fanyin3639/Ret…

Excited to share a new preprint on the 🩴FlipFlop Effect. We prompt LLMs with a classification task, and challenge the model by following up with “Are you sure?”. The model can confirm or flip its answer. The results? More flips than a gymnastics competition! 🤸‍♂️ 1/N

Excited to share this fun new work on the 🩴FlipFlop Effect. In short: if you ask models if they're sure of their answers, they tend to change their minds (and severely degrade accuracy). What's mindblowing is how universal the effect is across LLMs (GPTs, Gemini, Claudes, …).

Congratulations David Wan for this great collaboration between Salesforce Research and UNC! Shafiq Joty Mohit Bansal