It has not been reported much, but I believe ETH Zurich has, as of last week, banned new Master and PhD students who attended a long list of universities in China, Russia, and Iran. 🧵

Collection of insane and fun facts about SQLite. Let's go! SQLite is the most deployed and most used database. There are over one trillion (1000000000000 or a million million) SQLite databases in active use. It is maintained by three people. They don't allow outside

Why is Flash Attention so fast? Find out how Flash Attention works. Afterward, we'll polish our understanding by writing a GPU kernel of the algorithm in Triton. alexdremov.me/understanding-… #MachineLearning

New TMLR paper by Master (!) student @alexdremov_me: Training Dynamics of the Cooldown Stage in Warmup-Stable-Decay Learning Rate Scheduler We finally understand the negative square root (1-sqrt) cooldown. TL;DR: It gets you best bias-variance tradeoff :) w/ Atli Kosson, M Jaggi

New work from our MLO lab EPFL: Benchmarking the variety of different proposed LLM optimizers: Muon, AdEMAMix, ... all in the same setting, tuned, with varying model size, batch size, and training duration! Huge sweep of experiments by Andrei Semenov Matteo Pagliardini M Jaggi

This is one of my favorite results from our work on scaling laws for QAT. It helps answer the question: should you train an 8-bit model or a 4 bit model that has twice as many parameters? Both are the same size in RAM which is highly correlated with generation latency. Or even

The Maximal Update Parameterization (µP) allows LR transfer from small to large models, saving costly tuning. But why is independent weight decay (IWD) essential for it to work? We find µP stabilizes early training (like an LR warmup), but IWD takes over in the long term! 🧵

My favorite beatbox machine is my Mac struggling to play Spotify under 100% system load

8 8 6 2 And 2 being something like “the writing is not good enough”. Well, thanks for the valuable advice I guess

Well, we developed an intuitive framework for comparing different cooldown shapes (ps. there’s nothing special about the sqrt shape) arxiv.org/abs/2508.01483

there’s more and more reasons to submit your paper to TMLR 🙂