After 3 years of hard work, our unprecedented neural network training algorithm competition is finally open! The exciting part starts now, seeing what the community can create. 🏆Submit, become the next Adam, and bag $50,000 in prizes! mlcommons.org/2023/11/mlc-al…

A big thank you to Google AI for providing the compute resources for the MLCommons Algorithm benchmark efficiency competition. Learn more about how you can win some of the $50K prize money on our blog. mlcommons.org/2023/11/mlc-al…

Exciting! "The AlgoPerf: Training algorithms benchmark is a competitive, time-to-result benchmark that runs on a fixed system and compares training algorithms on multiple deep learning workloads. ... Submitters must develop and compete on the basis of more efficient algorithms."

Thrilled to share this work on materials discovery! We found that OOD generalization of GNNs improves predictably, with increasing data from quantum mechanical simulations. These GNNs allowed us to expand the number of known stable materials by an order of magnitude.

Improved training algorithms can save time, computational resources, and lead to better, more accurate, models. Thank you AI at Meta for participating in the MLCommons effort to drive innovation in training algorithms. Join the challenge!

Algorithm design is traditionally considered to be the most important type of work. Yet the story we keep seeing in modern ML is that the choice of hyperparameters matters at least as much as the choice of algorithm, and often more. IMO, the way to achieve lasting impact is

to focus less on inventing new algorithms, and to focus more on understanding the ones we already have.

Schedule-Free Wins AlgoPerf Self-Tuning Track 🎉 I'm pleased to announce that Schedule-Free AdamW set a new SOTA for self-tuning training algorithms, besting AdamW and all other submissions by 8% overall. Try it out: github.com/facebookresear…

The inaugural AlgoPerf results are in, highlighting a new generation of neural net training algorithms! Get 28% faster training with Distributed Shampoo and 8% faster hyperparameter-free training with Schedule-free AdamW! The future of training algorithms research is bright...

"Non-diagonal preconditioning has dethroned Nesterov Adam" 🧴👑 shampoo wins, finally the community can know what we have for years! this benchmark has been 3+ years in the making (we first talked about it Google in 2021), I'm beyond psyched that it's finally yielded results!

see this thread for an in-depth description of the benchmark setup, if you're interested in submitting your ideas you should attend the working group meetings on Thursdays, sign up here mlcommons.org/working-groups… 🚀🚀🚀 x.com/zacharynado/st…

Congratulations to everyone who submitted to the MLCommons AlgoPerf training algorithms competition! We were delighted to provide compute resources for evaluating so many exciting submissions.

In 2020, we introduced an AI method capable of generating superhuman chip layouts. Today, we describe its impact on the field and give it a name: AlphaChip!

We are also releasing a model checkpoint pre-trained on 20 TPU blocks, so our open-source method now has open-weights as well! Open-source repo: github.com/google-researc… Pre-training tutorial: github.com/google-researc… Pre-trained checkpoint: github.com/google-researc…

So, it turns out you don't get much time to throw together a Nobel Prize party, but we got people together to celebrate! 🎉 Here's me & Geoff before the party started, & a nice group photo of all of Geoff's former PhD students that were able to attend on short notice.

Moving to a new platform (same username), see the link in my profile.