🍷Imagine you are the boss of Google DeepMind. To train the best diffusion language model in world within 1 year, using 800 TPU pods, which model size will you go for? 🐿️ We build Quokka to help you decide–the first-ever large-scale scaling law for DLMs. Interesting facts: 1.

Happening tomorrow at 2:30pm ET / 11:30 am PT

🎓 Officially a doctor now 😊!!! As a first-gen college kid, this moment means the world to me. Grateful beyond words to all my mentors who’ve guided me along the way — from Georg Martius who first introduced me to research back in 2017, to Volodymyr Kuleshov 🇺🇦 who sparked my love for

We’re dropping “The Diffusion Duality, Chapter 2” soon! So, stay tuned 🤗

✨ Masked Generative Models (MGMs) are powerful and can generate tokens in parallel. They’ve driven impressive results across text and images and are increasingly competitive with autoregressive (AR) models. Thrilled to share our latest work to accelerate MGMs (1/12) 🧵

Funny enough, after we released MDLM last year, Sasha Rush came up with the exact same idea!

Impressive work by Justin Deschenaux ! They propose to replace the Encoder only denoising transformer with an Encoder-Decoder architecture which leads to faster training and inference of MDLM.

How do you even compute such probabilities?

Happy Diwali — from mine to yours ✨

✨Masked Diffusion Language Models✨ are great for reasoning, but not just for the reasons you think! Fast parallel decoding? 🤔 Any-order decoding? 🤨 Plot twist: MDLMs offer A LOT MORE for inference and post-training! 🎢🧵

Drowning in the sea of Discrete Diffusion papers? 🌊 We got you. Join our Reading Group! From theory → empirics, and language → molecules — we’ll decode the chaos together 💫 Join the cult—uh, I mean community 😇 👉 Google Group:  groups.google.com/g/diffusion-ll… (1 / 2)

Overwhelmed by the number of Diffusion LLM papers? 🌊 Same here 😭 So I’m starting a Discrete Diffusion Reading Group (Discrete Diffusion Reading Group) with my favorite disciples Justin Deschenaux and Zhihan Yang ✨ We’ll cover everything—from theory to empirics, from language to molecules. Join

We’re building a space that connects researchers, students, and practitioners working on discrete diffusion. Join the Discord — collaborate, learn, and share! Whether you’re 💼hiring or showcasing your work, this is the place 👇 Discord: discord.gg/JxSCwpNb

The term AGI gives me the same ick that “AI” did back in 2015. If it takes hundreds of billions of tokens just to get a respectable score on grade school math (GSM8K), that says everything about where we actually are.