Excited to continue building Modula in NumPy with Jeremy Bernstein ! Hope you can join us in-person or through zoom

Ethan Epperly typedfemale Lukas Balles rohan anil Yesterday in Scale ML we implemented autograd, so today I'm implementing modular normalization. I'll try to code up the Newton-Schulz iteration for Linear.normalize if there's time. Remember you can sign up for the zoom on atlas-mlc.github.io if you have an academic email

Today we welcome Nikhil Vyas to speak about his work on understanding the Shampoo Optimizer at 3pm! Sign up via our website scale-ml.org to join via zoom! Paper: arxiv.org/pdf/2406.17748…

After a mini break, we are back! Tomorrow we will have Eran Malach from Harvard giving a talk titled: Computational Benefits and Limitations of Transformers and State-Space Models For those interested in listening virtually, signup on our website!

Tomorrow, we will have Moshik Hershcovitch present: ZipNN - A Lossless Compression Library tailored for AI models. Join via zoom by signing up on our website! scale-ml.org

Excited to host Charlie Blake and Constantin tomorrow to talk about their work u-μP! Sign up via scale-ml.org to join remotely via Zoom!

This week we have Yuka Ikarashi (Yuka Ikarashi) present: Exocompilation for Specialized Hardware! Date/Time: Nov 13 3pm Sign up via scale-ml.org to join our mailing list for the zoom link

Hello everyone, this week at 3pm EST Nov 20 (Wed) we will be having Guangxuan Xiao present his work about efficient/effective long sequence modeling! Sign up via scale-ml.org to join our mailing list and zoom access.

For our last seminar of the year we will end with Lucas Wilkinson from @neuralmagic presenting! Machete: a cutting-edge mixed-input GEMM GPU kernel targeting NVIDIA Hopper GPUs Time: Dec 4, 3pm EST Sign up via scale-ml.org to join our mailing list for the zoom link

We are back with X. Dong from NVIDIA Research presenting Hymba, a state-of-the-art hybrid architecture that’s leading the way in the small language model revolution! Time: Jan 22, 4pm EST Sign up at scale-ml.org to join our mailing list for the zoom link

This week we will be having Boyuan Chen present his exciting work on diffusion models! Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion Time: Feb 5, 3pm EST, sign up at scale-ml.org to join our mailing list for the zoom link.

This week we will be having Yikang Shen present his exciting work on efficient training of LLMs! Date: Wednesday Feb 26, 4pm EST To attend the talk virtually, sign up via scale-ml.org MIT-IBM Watson AI Lab

We are excited to have Songlin Yang present: Linear Attention and Beyond 🚀🚀🚀 Time: Mar 5, 4pm EST, sign up at scale-ml.org to join our mailing list for the zoom link.

We are excited to have Heyi Tang from Kimi.ai present: Design and Optimization of Large-Scale Inference Systems of kimi.ai🚀 Time: Mar 12 7:15PM EST, sign up at scale-ml.org to join our mailing list for the zoom link.

This week we will be having a Yifei Zhou present his exciting work on: Self-improvement of LLM agents through Reinforcement Learning At Scale ⚡️ Time: Mar 26 12PM EST, sign up at scale-ml.org to join our mailing list for the zoom link.

We are exploring a new domain in Scale ML this week! Jeremy Wohlwend, will be presenting his exciting work on Boltz-1! Time: April 9th 4PM EST, sign up at scale-ml.org to join our mailing list for the zoom link.