✨ Very happy to announce I'm joining Hugging Face next week! 🤗 I'm based in the amazing Bern office 🇨🇭🏔️ alongside Leandro von Werra Lewis Tunstall Andi Marafioti working with the Language Model Dataset team to champion open datasets and models while supporting the open-science community.

Leandro von Werra so fast! just used it to figure out at which speed to go home and get less wet

The Ultra-Scale Playbook is REALLY good. Gives clear & specific info about LLM training tricks at scale as consecutive optimisation cascades. Gets a flow going. Some notes I made so far (very chill, non-rigorous, basically patching pre-existing info in my head):

MoE Sparsity over time: interesting to see the field starting quite conservative and then pushing more and more sparsity. Some observations: > Early models like Mixtral, DBRX, and Grok-1 went for ~25% sparsity across model sizes. > Llama 4 Scout and Maverick had the same

🇺🇦 Dzmitry Bahdanau Indeed, I mixed it up and thought it was the same as DeepSeek as they re-used the arch. Forgot that they change active/passive (32B/1T):

🎉 New chapter: just started at Hugging Face! Excited to help science teams bring their ML research to life through interactive demos and visualizations. Ready to dive in!

🚀 First week at Hugging Face: shipped improvements to The Ultra Scale Playbook! ✅ Dark mode ✅ Some mobile responsiveness ✅ Performance fixes Your complete guide to scaling #LLMs in 2025 👇 huggingface.co/spaces/nanotro…

AI for Science is one of the most exciting applications for AI. Accelerating the progress of science has potentially a far greater positive impact for society than yet another chatbot. There are so many startups popping up in the last 12 months tackling the most challenging and

The Ultra-Scale Playbook just got a beautiful update! - official dark mode - mobile compatible - more responsive overall You can order it as a book or get the PDF with a pro subscription. huggingface.co/spaces/nanotro… Not a dark mode fan but can't deny that it looks really cool:

If you need high quality data contact your friendly data-dealers of choice: Guilherme Penedo and Hynek Kydlíček

FineWeb has much more use cases than you can imagine! I would highlight the impact on multilingual training - we used FineWeb2 for first (and future👀) Ukrainian-focused MamayLM releases and it helped a lot🤗 Thanks to FW2 we will 100% see more multilingual models in the future!

Copy arrived today. I appreciate the practical coverage of data/tensor/pipeline/expert parallelism and profiling. Eager to read. Hugging Face, Leandro von Werra, Thomas Wolf

Finally got my copy of this book. Eager to read it! Thanks again Zach Mueller

LLM pretraining data processing has evolved over time: First: the more data the better with some light heuristic filtering (C4, gopher, redpajama, Pile, FineWeb, Dolma) Recently: remove all the bad data very aggressively with LLM classifiers but in return replay to train longer

Nano Banana is now available on Hugging Face Spaces for free for PRO users! 🤗 🤝🍌

Kimi's founder, Zhilin Yang's interview is out. Again, you can let Kimi translate for you: ) lots of insights there. mp.weixin.qq.com/s/uqUGwJLO30mR… Several takes: 1/ Base Model Focus: K2 aims to be a solid base model. We've found that high-quality data growth is slow, and multi-modal

Wow! Hugging Face reached 2M models! 🎉