We are excited to release Seahorse 🌊🐴, a ✨multilingual, multifaceted summarization evaluation dataset✨ 96,000+ human ratings to enable faster progress in training and evaluating learnt metrics for summarization! Preprint: arxiv.org/abs/2305.13194 Data: goo.gle/seahorse

Have you ever wanted a LangID model that works on 1500+ languages? check out FUN-LangID: github.com/google-researc… !

Introducing COLM (colmweb.org) the Conference on Language Modeling. A new research venue dedicated to the theory, practice, and applications of language models. Submissions: March 15 (it's pronounced "collum" 🕊️)

Excited to announce the First Conference on Language Modeling, to be held in approximately a year from now. Please let us know if you are interested or have any feedback on the conference: colmweb.org/survey.html

Seahorse (a multilingual summarization evaluation dataset with 96K+ human ratings) has been accepted to #EMNLP2023! See you there 🌊🐴 Bonus: The Seahorse metrics are now up on Hugging Face 🤗 hf.co/collections/go…

Excited to be presenting our work on **Evaluating and Modeling Attribution for Cross-Lingual Question Answering** at #EMNLP2023 in Singapore. Updated Paper: arxiv.org/abs/2305.14332 We're also releasing the XOR-AttriQA dataset: github.com/google-researc… 🧵

Our work on cross-lingual and multilingual attribution will be presented at #EMNLP2023 in Singapore! We have also released our dataset of ~10k 3-way annotations over 5 typologically diverse languages. Arxiv: arxiv.org/abs/2305.14332 Dataset: github.com/google-researc…

Excited to receive an Outstanding Paper award for this work at EMNLP 2025! Thanks to my co-authors George Foster and Markus Freitag! Updated version available here: aclanthology.org/2023.emnlp-mai…

Today at #NeurIPS2023, If you want to lean more about: 1. Robustness of detectors and watermarks to paraphrase attacks (spoiler alert: needs improvement). 2. An alternative detection approach using simple retrieval methods. and ...

ACL has removed the anonymity period. This means that ACL submissions can be posted and discussed online at any time, although extensive PR is discouraged. aclweb.org/adminwiki/imag…

New paper alert! Designing reliable human evaluation is both crucial and difficult. Human raters can exhibit different behaviors when rating NLG outputs. These differences are not generally due to a rater performing the task incorrectly, but rather due to differences in

Absolutely unhinged. When Jason Baldridge started this in 2021 he would enthusiastically show us weird new images that he took. I thought it was just some weird phase that would fizzle out, but I'm very happy to be wrong, and that it resulted in such a high quality dataset!

I will be at NAACL in-person, and co-organizing the AmericasNLP workshop there! #NLProc #NAACL2024

Excited to announce that 110 languages got added to Google Translate today! Time for context on these languages, especially the communities who helped a lot over the past few years, including Cantonese, NKo, and Faroese volunteers. Also, a 110-language youtube playlist. 🧵

As part of Google's 1,000 Languages Initiative, a commitment to support the 1,000 most spoken languages, & w/help of our PaLM 2 LLM, we're adding support for 110 new languages (spoken by 614M people) to Google Translate (now supporting 243 languages). 🎉 blog.google/products/trans…

Exciting news! As of today, the Inuit language of Inuktut will be available on Google Translate - marking the first Canadian Indigenous language on the platform. Tunngasugit! | ᑐᙵᓱᒋᑦ (Welcome!) 🎉 Huge thanks to ITK for their invaluable guidance and

🚀 Join the Gemini Multilinguality team Google DeepMind 🌐 We’re looking for researchers passionate about making LLMs helpful for all. Dramatically improve model quality, coverage, and cultural relevance across hundreds of languages. #NLProc #MultilingualAI #i18n #LLMs

Catch our Google Translate Research team at #EMNLP #WMT24! The team will present 9 papers on step-by-step decoding, mitigating metric bias within MBR decoding (+ MBR dataset release), improved human data collection and automatic metrics (MetricX: winner of WMT Metrics Task).

Today, we have released a benchmark and a corresponding leaderboard titled "FACTS Grounding" in collaboration with Kaggle. 1/

DumPy: Like NumPy except it's OK if you're dum