I am at ICML 2024 (ICML Conference) in Vienna. Come to our poster #1209 (today, 1:30pm-3pm, Hall C 4-9) on theory for Independent Subnetwork Training (IST). Led by my PhD student Egor Shulgin (Egor Shulgin); who is also here this week!

Gluon is an LMO-based optimizer that unifies Muon and Scion, addressing two key gaps 𝘪𝘯 𝘵𝘩𝘦𝘪𝘳 𝘵𝘩𝘦𝘰𝘳𝘺: 1. Layer-wise updates (not full-model) — aligns with how models are trained in practice 2. New smoothness model — uses (L₀, L₁)-smoothness per layer, not

Many many many thanks to the lecturers of the first week for their dedication (could only get some of them on this photo...) #MlssSenegal2025 Adji Bousso Dieng Mathieu Blondel Gergely Neu Ashia Natalie Schluter Marieme Ngom Elvis Dohmatob Egor Shulgin jo officer Eugene Ndiaye

I am organizing "Workshop on Distributed Training in the Era of Large Models" at KAUST during November 24-26, 2025. If you have done some cool work in the area, you might want to attend. The talks are invite-only. I'll soon start sending invites. More info later!