We just released a new approach for turning local search heuristics used to solve NP-hard combinatorial problems in OR into differentiable layers. The key idea is to use the neighborhoods used by these algorithms for creating MCMC proposal distributions arxiv.org/abs/2505.14240

Our paper derives momentum schedules that are functions of both the model dimension and data distribution. * On our theoretical model, this provably improves the scaling law exponents in many regimes! * And, this exponent improvement holds on LSTM experiments on C4.

Announcing Ambient Diffusion Omni — a framework that uses synthetic, low-quality, and out-of-distribution data to improve diffusion models. State-of-the-art ImageNet performance. A strong text-to-image results in just 2 days on 8 GPUs. Filtering ❌ Clever data use ✅

Pleased to see that this time, three Czech ladies are in the list of the European Research Council (ERC) Advanced Grants, and I am very proud to be among them ;). Congrats to Kateřina Čapková a Anna Durnová!

🎉 It’s official! I’ve been awarded an ERC Advanced Grant for my project on Statistical Analysis of Generative Models. More details 👉 crest.science/arnak-dalalyan… #ERCAdG European Research Council (ERC)

❓ How long does SGD take to reach the global minimum on non-convex functions? With Franck Iutzeler, J. Malick, P. Mertikopoulos, we tackle this fundamental question in our new ICML 2025 paper: "The Global Convergence Time of Stochastic Gradient Descent in Non-Convex Landscapes"

Looking back on an inspiring day of exchange last week at Mila, where students presented cutting-edge research work to their peers during a casual poster session. See you in September for the next edition!

It's hard to plan for AGI without knowing what outcomes are even possible, let alone good. So we’re hosting a workshop! Post-AGI Civilizational Equilibria: Are there any good ones? Vancouver, July 14th Featuring: Joe Carlsmith Richard Ngo Emmett Shear 🧵

🧵(1/6) Delighted to share our ICML Conference 2025 spotlight paper: the Feynman-Kac Correctors (FKCs) in Diffusion Picture this: it’s inference time and we want to generate new samples from our diffusion model. But we don’t want to just copy the training data – we may want to sample

New paper on the generalization of Flow Matching arxiv.org/abs/2506.03719 🤯 Why does flow matching generalize? Did you know that the flow matching target you're trying to learn **can only generate training points**? with Quentin Bertrand, Anne Gagneux & Rémi Emonet 👇👇👇

Yet FM generates new samples! An hypothesis to explain this paradox is target stochasticity: FM targets the conditional velocity field i.e. only a stochastic approximation of the full velocity field u* *We refute this hypothesis*: very early, the approximation almost equals u*

We propose to regress directly against the optimal (deterministic) u* and show that it never degrades the performance On the opposite, removing target stochasticity helps generalizing faster.

🚀IVADO et le Centre des Compétences futures Future Skills Centre - en collaboration avec le Tech3Lab de HEC Montréal, lancent une nouvelle formation gratuite en #IA pour les professionnel(le)s du #Québec et du #Canada. Lire le communiqué de presse➡️lnkd.in/gdDkqj8N

Yes! Indeed, deep generative networks do not exactly reproduce the training set/generalize because of the inductive bias. The key difference with prev. gen. models (e.g. GANs) is the closed-form formula of FM: one can study very finely where the inductive bias comes into play!

Mathurin Massias Quentin Bertrand Fascinating. So the failure to perfectly learn the target velocity field, due to neural network inductive biases, is what drives generalization in Flow Matching. Quite a reminder that sometimes the best solutions emerge from inherent system constraints

ResNet and Neural ODEs are closely related: ResNet uses discrete residual/skip connections, while Neural ODEs generalize this to continuous transformations using ODEs. Neural ODEs *can* be seen as the limit of ResNet as the number of layers approaches infinity.

Mayavi: 3D scientific data visualization and plotting in Python docs.enthought.com/mayavi/mayavi/

Burny - Effective Omni Mathurin Massias Quentin Bertrand Yes, it is basically a different way of training normalizing flows via a regressive objective on the vector field, thus avoiding simulation step a training time. Meta uses it!

Mila's science communication contest finale showcased 6 brilliant researchers pioneering AI for medical imaging, assistive robotics, forest monitoring, inclusive urban design and more. Watch the presentations that won the hearts of the jury and the public: ow.ly/wyIY50We2p3

Slides of my talk on our ICML 2025 paper "Joint Learning of Energy-based Models and their Partition Function" mblondel.org/talks/?p=ebm.m…