For large-scale causal discovery, there's no need to use NOTEARS for its speed. Consider using LiNGAM. We've parallelized it, achieving a 32x speed-up on GPUs. NOTEARS: Scalable: ✅ Identifiability guarantees: ❌ AcceleratedLiNGAM: Scalable: ✅ Identifiability guarantees: ✅

Introducing Unified PDE Solvers (UPS), a step towards efficiently building foundation models for PDE solvers (arxiv.org/abs/2403.07187)! Starting from a pretrained LM, UPS tackles diverse spatiotemporal PDEs with SOTA accuracy, using ~20x less data and a single A6000! 🧵[1/x]

I'm super excited to release DiPaCo, a new kind of mixture of experts, that can scale engineering-wise to data centers across the entire world! A few words about it in this thread 🧵

Ever noticed how Pixar adapts movies for international markets? The beloved newscaster in Zootopia is a jaguar in Brazil, a panda in China, a koala in Australia … While machine translation (MT) has only dealt with language in speech/text thus far, we extend the scope of MT to

Checkout our work on adapting multitask learning as a tool against worst case group error. Our modified MTL approach (main task + pre-training auxiliary objective + L1 embedding reg) is competitive against bespoke DRO (Distributionally Robust Optimization) methods

Are you interested in working with us on modularity and continual learning? Consider applying to our open full-time RE position in NYC: boards.greenhouse.io/deepmind/jobs/…

🚨 Interested in synthetic data and LLM reasoning? Our new work studies scaling laws for synthetic data and RL for math reasoning. TLDR: Step-level RL (per-step DPO in fig) on self-generated answers improves sample efficiency of synthetic data by 8x! arxiv.org/abs/2406.14532 1/🧵

Have you wondered how the decision boundary of in-context learning in LLMs compares to traditional models like Decision Trees and KNN? 🤔 Our research uncovers unexpected irregularities and non-smoothness in LLMs' in-context decision boundaries. 🔍 📄: arxiv.org/abs/2406.11233

Applications now officially open until end of month x.com/cohereforai/st… Looking forward to hearing from all of you!

Thank you so much EMNLP 2025 for this wonderful recognition! I’m so honored and humbled 💕 Thanks Graham Neubig for your support throughout! We’ve been working on this for 1.5 years and everyone who has spoken with me in the recent past knows how passionately I feel about this

I’m hiring PhD students in computer science at Columbia! Our lab will tackle core challenges in understanding and controlling neural models that interact with language. for example, - methods for LLM control - discoveries of LLM properties - pretraining for understanding

it is hard to overstate how much alec radford has contributed to the field, and how much of everyone's current progress traces back to his work. i believe he is a genius at the level of einstein, and also he is one of my favorite people ever--hard to imagine a nicer, warmer, or

1.5 yrs ago, we set out to answer a seemingly simple question: what are we *actually* getting out of RL in fine-tuning? I'm thrilled to share a pearl we found on the deepest dive of my PhD: the value of RL in RLHF seems to come from *generation-verification gaps*. Get ready to🤿!

The future of intelligence will be distributed … and DiLoCo may just be the 🔑 ingredient. We develop scaling laws for DiLoCo with many interesting findings — see 🧵 Work led by Zachary Charles — such a fun colab —cheers to many more.

Hi Friends ! I'll be in Singapore for ICLR 2025. Would be great to catchup and talk about research :) I'm interested in most things ML but my current primary focus has been : - model self-improvement - continual learning and modularity - distributed / collaborative learning

Are you a frontier lab investing untold sums in training? Are you trying to stay competitive? Are you finding that your competitors' models are ... thinking a bit too much like yours? Then antidistillation.com might be for you! Sam Altman Elon Musk

30+ accepted papers 6 oral papers 6 guest speakers join us at ICLR 2026 on the 27th Hall 4 #3 for a full day of workshop on Modularity for Collaborative, Decentralized, and Continual Learning sites.google.com/corp/view/mcdc… Lucio Dery Jnr Mwinm, Fengyuan Liu, and myself will be organizing

Come to our workshop !!! It’ll be fun — I promise !!

1/ With Benjamin Laufer and Jon Kleinberg, we constructed the largest dataset of its kind to date: 1.86M Hugging Face models. In a new paper, we mapped how the open-source AI ecosystem evolves by tracing fine-tunes, merges, and more. Here's what we found 🧵

🥹🥹 All vague-posting aside, super happy this model is finally out there & proud of everyone for making this happen 💖 let us know what you think!