🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Great article on Model Collapse out in the New York Times! Features great work (Sina Alemohammad Ilia Shumailov🦔 Zakhar Shumaylov Nicolas Papernot Quentin Bertrand et al) and some of ours with Elvis Dohmatob Yunzhen Feng François Charton Pu Yang 杨 埔 Kudos to Aatish for a super job nytimes.com/interactive/20…

Some exciting news coverage of iterative retraining of generative models by the NYT! nytimes.com/interactive/20… The article includes several papers including some of my co-authors from Mila - Institut québécois d'IA Quentin Bertrand Damien Ferbach Gauthier Gidel were excited to write: Broad strokes

I am delighted to share that our paper has been accepted at #NeurIPS as a spotlight!🚀 A huge thanks to my amazing collaborators Quentin Bertrand, Joey Bose and my supervisor Gauthier Gidel !!

The slides can be found here! qb3.github.io/pdfs/iterative…

Dreaming of pushing the boundaries in AI? 🌟 Apply for MSc or PhD at Mila, a world-leading research hub, for Fall 2025! 🚀 #AI #Research #Mila #PhD #MSc

I am hiring Ph.D. and/or M.Sc. students to work at the intersection of game theory, optimization, and Machine Learning in Fall 2025. If you are interested in working in my group, apply via mila.quebec/en/prospective… More details on the topics in the 🧵👇 1/n

My group will be presenting a paper a NeurIPS on its topic: arxiv.org/pdf/2407.09499 Lead by Damien Ferbach with the amazing Quentin Bertrand and Joey Bose 6/n

I will be at NeurIPS in Vancouver this week to present our work on self-consuming generative models. arxiv.org/abs/2407.09499 Please reach out to talk about high-dimensional optimization and generative models !

Come checkout our spotlight poster in the East ballroom #2510 happening now with Damien Ferbach Quentin Bertrand Gauthier Gidel Especially come if you're interested in self Consuming generative models and model collapse!!

Come checkout our poster now!!! Poster 2510 east ballroom. Joey Bose Gauthier Gidel Quentin Bertrand Mila - Institut québécois d'IA

Un aperçu du poster spotlight de Damien Ferbach présenté à NeurIPS Conference, Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferences:

New Paper! 📄 Once a model memorizes an example, it stops learning from it! Our latest work explores this phenomenon and the nuanced interplay between memorization and generalization. Let’s dive in! 🚀🧵

1. We often observe power laws between loss and compute: loss = a * flops ^ b + c 2. Models are rapidly becoming more efficient, i.e. use less compute to reach the same loss But: which innovations actually change the exponent in the power law (b) vs change only the constant (a)?

Hestness et al 2017 asked this question and concluded "model improvements only shift the error but do not appear to affect the power-law exponent". Are we doomed to chase constant factor improvements? Is there anything we can do to improve the exponent?

In summary, while data can change the power law exponent significantly, it's quite challenging to find architectures and optimizers that change the exponent! If you want to see something that *does* change the power law exponent, stay tuned and follow Damien Ferbach for more 😁