Pretty strong results. I was always wondering why not design a metric that works for a specific test set. Here we go!

We just open-sourced MetricX-24, our SOTA automatic metric for MT with clear improvements over MetricX-23. One of the highlight is is language coverage and how well it performs even for language pairs it was not fine-tuned for!

Highly recommend everyone to read Yusuf's work on data contanimation! He trained almost 90 models investigating how different types of contanimation affect test set scores! Very insigthful paper.

🚨New machine translation dataset alert! 🚨We expanded the language coverage of WMT24 from 9 to 55 en->xx language pairs by collecting new reference translations for 46 languages in a dataset called WMT24++ Paper: arxiv.org/abs/2502.12404… Data: huggingface.co/datasets/googl…

😼SMOL DATA ALERT! 😼Anouncing SMOL, a professionally-translated dataset for 115 very low-resource languages! Paper: arxiv.org/pdf/2502.12301 Huggingface: huggingface.co/datasets/googl…

.Google and Imperial College London researchers argue that translation quality can’t be captured 🛑 by a single metric and propose evaluating #AI #translation systems 🤖 on an accuracy-naturalness plane. #xl8 #t9n #Google Google AI Markus Freitag slator.com/google-calls-f…

MT metrics excel at evaluating sentence translations, but struggle with complex texts We introduce *TREQA* a framework to assess how translations preserve key info by using LLMs to generate & answer questions about them arxiv.org/abs/2504.07583 (co-lead Sweta Agrawal) 1/15

Every month, people use Google to translate around 1 trillion words. Today, we’re introducing a new AI-powered live translation experience in the Google Translate app, plus a new beta feature to help you practice new languages. Rolling out now on iOS + Android.