🚀 Excited to announce EuroBERT: a new multilingual encoder model family for European & global languages! 🌍 🔹 EuroBERT is trained on a massive 5 trillion-token dataset across 15 languages and includes recent architecture advances such as GQA, RoPE & RMSNorm. 🔹

🧵 (3/7) 🌐 EuroBERT is open-source: 👉 Models (210M, 610M, 2.1B params) 👉 Training snapshots 👉 Full training framework Explore here: [huggingface.co/EuroBERT](huggingface.co/EuroBERT) Code coming soon! [github.com/Nicolas-BZRD/E…](github.com/Nicolas-BZRD/E…)

🧵 (5/7) Nicolas Boizard Hippolyte Gisserot-Boukhlef Andre Martins Ayoub Hammal Caio Céline Hudelot, Emmanuel Malherbe, Etienne Malaboeuf Fanny Jourdan Gabriel Hautreux João Alves Kevin El-Haddad Manuel Faysse Maxime Peyrard Nuno M. Guerreiro Patrick Fernandes Ricardo Rei Pierre Colombo

🧵 (7/7) 📖 Check out our blog post for more insights: huggingface.co/blog/EuroBERT/… 📄 Read more in our paper: arxiv.org/abs/2503.05500

🚨 Introducing EuroBERT, a family of multilingual encoder models (210M to 2.1B params) trained on 5T tokens with a 8,192 sequence length and all the modern bells and whistles! It's open source, and hopefully the perfect base model to train multilingual embeddings ! (1/N)

Here is a blogpost with more details: huggingface.co/blog/EuroBERT/… Work led by my amazing PhD colleagues Nicolas Boizard Hippolyte Gisserot-Boukhlef Duarte Alves Pierre Colombo (5/N)

More encoder upgrade and multilingual this time (so no excuse not to try it)! Great work from the team, have been in some discussions regarding these models and was really looking forward to the release! 🚀 Congratulations Nicolas Boizard and Manuel Faysse!

More BERTs for the Modern era. This is super exciting, encoders are no longer dead 😄 The coolest aspect: with so many new proofs that encoders are small-param-count beasts, this'll hopefully spark a lot more research onto making them even better in creative ways...

An assembly of 18 European companies, labs, and universities have banded together to launch 🇪🇺 EuroBERT! It's a state-of-the-art multilingual encoder for 15 European languages, designed to be finetuned for retrieval, classification, etc. Details in 🧵

Models: huggingface.co/EuroBERT

This is pretty cool! If people only knew how much we still rely on encoder-only models in AI products! 🔥

EuroBERT: Scaling Multilingual Encoders for European Languages

EuroBERT is out and it's insane! 🇪🇺 It's the most powerful multilingual encoder model family at SOTA across a wide range of tasks: retrieval, classif, regression, maths, and code. 3 sizes: 210M, 610M, and 2.1B parameters, with support sequence lengths up to 8,192 tokens. 📖 ⤵️

🤩 For more details, check out Nicolas Boizard' excellent thread: x.com/N1colAIs/statu…

[EuroBERT: Multilingual Embedding Model] 🔥 There is a new open embedding model in town - EuroBERT claims superior performance across a diverse set of benchmarks, spanning multilingual capabilities, mathematics, and coding. It even outperforms ModernBERT on code and math

Not just yet another LLM 🇪🇺 EuroBERT provides open multilingual encoder models to power retrieval, classification & embeddings across 15 languages. But it is yet another model trained on AMD compute platforms. 🚀 European companies, labs, and universities have made this come

More info there: huggingface.co/blog/EuroBERT/…

Great to see the community using EuroBERT! As hoped, it’s proving to be an excellent foundation model, especially for information retrieval tasks across multiple languages after just one epoch of finetuning. Check it out: huggingface.co/Omartificial-I…: Eng.Omar

Eng.Omar *huggingface.co/Omartificial-I… congratulations: Eng.Omar

🚨We are moving Visual Document Retrieval Evaluation to MTEB! Starting today, ViDoRe V1 and V2, but soon joined by many other benchmarks will benefit from first-class support in MTEB, enabling adding models and tasks more easily and more collaboratively! More in 🧵(1/N)