on my way back to NYC, i met wise Leon Bottou in the airport. we talked. then i told him "you should tweet that!" and, he delivered much more than a tweet: a blog post with thoughts and insights on AI research only he can deliver this clearly and succinctly.

Henry Shugart, Jason M. Altschuler: Negative Stepsizes Make Gradient-Descent-Ascent Converge arxiv.org/abs/2505.01423 arxiv.org/pdf/2505.01423 arxiv.org/html/2505.01423

Check out this excellent work led by Dmitry Dmitry Rybin ! We discovered a new algorithm to compute the matrix product XX^t with 5% fewer number of multiplications

🧐This is definitely worth reading. I've been puzzling with this phenomenon for quite a while

man, scientists working on optimizing matrix multiplications have oppenheimer level of aura - use a RL agent to spit out heckload of bilinear products - slap two MILP to combine and filter those - iterate on top of a Large Neighborhood Search flow until it’s fast fast what the

Dear Professors who are running the ICML board election, single column please, and we will definitely vote for you 😀 #icml2025 #ICML

🚀 Hello, Kimi K2! Open-Source Agentic Model! 🔹 1T total / 32B active MoE model 🔹 SOTA on SWE Bench Verified, Tau2 & AceBench among open models 🔹Strong in coding and agentic tasks 🐤 Multimodal & thought-mode not supported for now With Kimi K2, advanced agentic intelligence

Holy shit. Kimi K2 was pre-trained on 15.5T tokens using MuonClip with zero training spike. Muon has officially scaled to the 1-trillion-parameter LLM level. Many doubted it could scale, but here we are. So proud of the Moum team: Keller Jordan, Vlado Boza, You Jiacheng,

Awesome! Kaiyue Wen this is related to our discussion before.

I built an interactive visualization tool to understand how MuonClip helped with Kimi.ai 's K2 training! Try it yourself: …-app-644257448872.us-central1.run.app (1/11)

[1/9] We created a performant Lipschitz transformer by spectrally regulating the weights—without using activation stability tricks: no layer norm, QK norm, or logit softcapping. We think this may address a “root cause” of unstable training.

7.21 - 7.24 at ICCOPT #ICCOPT Happy to chat😃

hhhhhhhhh