"DataPerf: Benchmarks for Data-Centric AI Development" What if instead of holding the data constant and benchmarking different models, we held the model constant and benchmarked different data pipelines? [1/7]

We are very excited to be teaming up with MLCommons! Stay tuned for our upcoming new datasets and other updates!

How can we improve benchmarking? The Dynabench experiment aims to make faster progress with dynamic data collection, and today, we are pleased to introduce our next stage: @MetaAI has funded 5 exciting research proposals on the theme of "Rethinking Benchmarking”! Congrats to:

Ever wanted to know more about generalisation in NLP but overwhelmed with the number of papers on ArXiv? Fear not! We read 400+ papers, 600+ experiments, and designed a taxonomy 📝 to categorise the research for you! (1/n) 🧵 arxiv.org/abs/2210.03050

Great talk today by Max Bartolo from UCL Natural Language Processing @CohereAI on Dynamic Adversarial Data Collection at the KCL Informatics Distributed AI group!

Announcing the BabyLM 👶 Challenge, the shared task at CoNLL 2025 and CMCL'23! We’re calling on researchers to pre-train language models on (relatively) small datasets inspired by the input given to children learning language. babylm.github.io arxiv.org/abs/2301.11796

Today, Meta researchers together with MLCommons working group, are launching DataPerf, the first platform for building data & data-centric AI algorithm leaderboards. We're excited for how DataPerf will help to push the data-centric AI field forward ⬇️

Big congrats to longterm Dynabenchers Douwe Kiela and Amanpreet Singh on their exciting launch! Looking forward to seeing what you build! 🎉

Progress in AI continues to outpace benchmarks. Check out this new plot, inspired by Dynabench, that shows just how quickly it's happening. Read more about it here: contextual.ai/plotting-progr…