New #FeaturedCertification: WOODS: Benchmarks for Out-of-Distribution Generalization in Time Series Jean-Christophe Gagnon-Audet, Kartik Ahuja, Mohammad Javad Darvishi Bayazi et al. openreview.net/forum?id=mvftz… #generalization #generalize #datasets

*Model Ratatouille: Recycling Diverse Models for OOD Generalization* by Alexandre Ramé Kartik Ahuja Matthieu Cord Creating robust models by averaging the weights of multiple models fine-tuned from the same initialization can be a very effective strategy. arxiv.org/abs/2212.10445

Tired of brittle AI models? Our paper “Context is Environment” shows that in-context learning in LLMs holds the key to domain generalization. Introducing In-Context Risk Minimization (ICRM)! Paper: arxiv.org/pdf/2309.09888…

In-context learning meets domain generalization (DG)! 1. DG researchers should consider environment as context to build better generalizers. 2. Train a transformer on a sequence of unlabelled data from the domain followed by the current query to predict its label.

Exciting news 🎓! I'm defending my PhD on "Diverse & Efficient Ensembling of Deep Networks" tomorrow at 13h30 CEST. If you're in Paris and can join, DM me. Or catch the live stream on YouTube: youtube.com/watch?v=DTD7qt…. Wish me luck!

Dhanya Sridhar, Chandler Squires and I are running a talk series on Causality, Abstraction, Reasoning and Extrapolation (CARE).First talk is at 11EST tomorrow by Johnny on "The Statistical Structure of Identifiable Generative Models".All are welcome! portal.valencelabs.com/events/post/th…

This Wed. 10:30 ET (29th Nov.) @ CDG we are honored to see Divyat Mahajan with their CLeaR-Conference on Causal Learning and Reasoning paper: Towards efficient representation identification in SL (proceedings.mlr.press/v177/ahuja22a/…) All info & links @ discuss.causality.link 🌿 cc: Kartik Ahuja Vasilis Syrgkanis Ioannis Mitliagkas (Γιάννης Μητλιάγκας)

Vote for me for a Leadership in Responsible AI award. linkedin.com/feed/update/ur…

Congrats Julius von Kügelgen and team! Great work :)

Better late to the party than never. We have 3 works at #NeurIPS2023 , but sadly I’m not able to present any of them as the visa officer didn’t care to even look at my invitation/presentations/paper/etc the moment he learned about my nationality. (cont. in the thread)

Congrats to Jean-Christophe Gagnon-Audet Kartik Ahuja @653mjd and Guillaume Dumas - our paper on OoD time-series benchmarks (under-researched modality when studying OoD) got featured certification by TMLR and will be also presented at ICLR in May! See you in the WOODS ;) openreview.net/forum?id=mvftz…

Explore fundamental questions surrounding large language models by applying for a postdoctoral position with our Generalization Team at FAIR Paris. If you're attending ICLR, visit our booth to connect with my colleagues and learn more about this exciting opportunity.

This work delivers on both theory and practice—offering the sharpest provable compositionality guarantees I know of, alongside state‑of‑the‑art performance on tough compositional distribution‑shift benchmarks.

1/Excited to share the first in a series of my research updates on LLM pretraining🚀. Our new work shows *distilled pretraining*—increasingly used to train deployable models—has trade-offs: ✅ Boosts test-time scaling ⚠️ Weakens in-context learning ✨ Needs tailored data curation

Distilled pretraining isn’t just a trend—it is the future of pretraining. Scratch trained models are on their way out (see Llama 4 Maverick, Gemma 3). Yet the science is still wide open. Amazing work here by our intern Sachin Goyal.

A very intriguing piece of work from colleagues at FAIR Paris.

🚨 Super excited to finally share our Safety Pretraining work — along with all the artifacts (safe data, models, code)! In this thread 🧵, I’ll walk through our journey — the key intermediate observations and lessons, and how they helped shape our final pipeline.