There exists f:[0,1]→[0,1] strictly increasing, continuous function such that its derivative is 0 almost everywhere. jstor.org/stable/2978047…

🏆 #Distinction | Toutes nos félicitations à Gérard Biau (Centre Inria Sorbonne Université), directeur de #SCAI et spécialiste des dynamiques statistiques dans les algorithmes d'IA, qui a été élu à l’Académie des sciences 👏. sorbonne-universite.fr/presse/gerard-…

🍏🍏🍏 Come work with us at Apple Machine Learning Research! 🍏🍏🍏 Our team focuses on curiosity-based, open research. We work on several topics, including LLMs, optimization, optimal transport, uncertainty quantification, and generative modeling. Infos 👇

When optimization problems have multiple minima, algorithms favor specific solutions due to their implicit bias. For ordinary least squares (OLS), gradient descent inherently converges to the minimal norm solution among all possible solutions. fa.bianp.net/blog/2022/impl…

My book is (at last) out, just in time for Christmas! A blog post to celebrate and present it: francisbach.com/my-book-is-out/

The Mathematics of Artificial Intelligence: In this introductory and highly subjective survey, aimed at a general mathematical audience, I showcase some key theoretical concepts underlying recent advancements in machine learning. arxiv.org/abs/2501.10465

Our work on geometric disentangled representation learning has been accepted to ICLR 2025! 🎊See you in Singapore if you want to understand this gif better :)

Learning rate schedulers used to be a big mistery. Now you can just take a guarantee for *convex non-smooth* problems (from arxiv.org/abs/2310.07831), and they give you *precisely* what you see in training large models. See this empirical study: arxiv.org/abs/2501.18965 1/3

Learning rate schedules seem mysterious? Turns out that their behaviour can be described with a bound from *convex, nonsmooth* optimization. Short thread on our latest paper 🚇 arxiv.org/abs/2501.18965

A really fun project to work on. Looking at these plots side-by-side still amazes me! How well can **convex optimization theory** match actual LLM runs? My favorite points of our paper on the agreement for LR schedules in theory and practice: 1/n

Optimization algorithms come with many flavors depending on the structure of the problem. Smooth vs non-smooth, convex vs non-convex, stochastic vs deterministic, etc. en.wikipedia.org/wiki/Mathemati…

It was received quite enthusiastically here so time to share it again!!! Our #ICLR2025 blog post on Flow M atching was published yesterday : iclr-blogposts.github.io/2025/blog/cond… My PhD student Anne Gagneux will present it tomorrow in ICLR, 👉poster session 4, 3 pm, #549 in Hall 3/2B 👈

📣 New preprint 📣 **Differentiable Generalized Sliced Wasserstein Plans** w/ L. Chapel Romain Tavenard We propose a Generalized Sliced Wasserstein method that provides an approximated transport plan and which admits a differentiable approximation. arxiv.org/abs/2505.22049 1/5

🧵 I'll be at CVPR next week presenting our FiRe work 🔥 TL;DR: We go beyond denoising models in PnP with more general restoration (e.g. deblurring) models! A starting point observation is that images are not fixed-points of restoration models:

❓ How long does SGD take to reach the global minimum on non-convex functions? With Franck Iutzeler, J. Malick, P. Mertikopoulos, we tackle this fundamental question in our new ICML 2025 paper: "The Global Convergence Time of Stochastic Gradient Descent in Non-Convex Landscapes"

I want to address one very common misconception about optimization. I often hear that (approximately) preconditioning with the Hessian diagonal is always a good thing. It's not. In fact, finding a good preconditioner is an open problem, which I think deserves more attention. 1/4

Back from MLSS Senegal 🇸🇳, where I had the honor of giving lectures on differentiable programming. Really grateful for all the amazing people I got to meet 🙏 My slides are here github.com/diffprog/slide…

For evolving unknown PDEs, ML models are trained on next-state prediction. But do they actually learn the time dynamics: the "physics"? Check out our poster (W-107) at #ICML2025 this Wed, Jul 16. Our "DISCO" model learns the physics while staying SOTA on next states prediction!

🚟 New blog post: On "infinite" learning-rate schedules and how to construct them from one checkpoint to the next fabian-sp.github.io/posts/2025/09/…

Nesterov dropped a new paper last week on what functions can be optimized with gradient descent. The idea is simple: we know GD can optimize both nonsmooth (bounded grads) and smooth (Lipschitz grads) functions, but smooth+nonsmooth satisfies neither property, so what can we do?