New paper alert: Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptation arxiv.org/abs/2410.08305 #LLM #LLMs #PEFT #LoRA LoRA is the arguably the most successful method for parameter-efficient fine-tuning (PEFT) of large language

I am hiring Ph.D. and/or M.Sc. students to work at the intersection of game theory, optimization, and Machine Learning in Fall 2025. If you are interested in working in my group, apply via mila.quebec/en/prospective… More details on the topics in the 🧵👇 1/n

(1/4) Excited to share our NeurIPS 2024 paper: Remove that Square Root: A New Efficient Scale-Invariant Version of AdaGrad 🚀 🤝work with Nazarii Tupitsa Nicolas Loizou Samuel Horváth Martin Takac Eduard Gorbunov 📍Poster: West Ballroom A-D #6103 📅 Wed 11 Dec 11 a.m.- 2 p.m. PST

I am not attending NeurIPS Conference this year but many of my students are! Do reach out to Omar Shaikh Omar, Hanmin Li, Kai Yi (Kai Yi), Abdurakhmon Sadiev (Abdurakhmon Sadiev), Kaja Gruntkowska, Egor Shulgin (Egor Shulgin), Artavazd Maranjyan and Marta Pozzi! Here are our papers:

Congratulations to my PhD student Kaja Gruntkowska for winning the 2024 Al-Kindi Research Award!

Excited to invite you to my Spotlight poster at #NeurIPS2024! My co-author Artem Agafonov will talk about “Exploring Jacobian Inexactness in Second-Order Methods for Variational Inequalities: Lower Bounds, Optimal Algorithms, and Quasi-Newton Approximations”. 📌 West Ballroom

Can we achieve both strong optimization guarantees and rigorous privacy protection at the same time? Absolutely! In our latest paper, we introduce Clip21-SGD2M, a powerful new method that effectively tackles both stochastic and DP noises.

📢: The 124th FLOW talk is on Wednesday (April 9) at **1 pm UTC.** Please register for our mailing list: bit.ly/3WVplLU. FLOW Calendar: bit.ly/3E9iIT4.

I’m pleased to share that, starting August 1, I will be joining MBZUAI (MBZUAI) as an Assistant Professor in the Department of Statistics and Data Science. My research focuses on optimization for machine learning, with an emphasis on stochastic methods, federated learning,

🚀 Join @MBZUAI (Abu Dhabi) to advance efficient & distributed ML! 🔹 Postdoctoral Associate — apply.interfolio.com/170202 🔹 Visiting Student — apply.interfolio.com/170203 Apply now and help shape the future of #MachineLearning. #AI #Postdoc #PhD #Careers

Looking for postdocs to join my Optimization & Machine Learning Lab @ KAUST! Write to me by the end of November 2025 if you want to apply. richtarik.org richtarik.org/i_papers.html richtarik.org/i_team.html richtarik.org/i_join.html

I am organizing "Workshop on Distributed Training in the Era of Large Models" at KAUST during November 24-26, 2025. If you have done some cool work in the area, you might want to attend. The talks are invite-only. I'll soon start sending invites. More info later!