Excited to share our new paper on how SGD biases towards simpler models via stochastic collapse to the invariant sets.

1/ Our new paper lead by Allan Raventós Mansheej Paul , FENG CHEN asks when in-context learning of regression can solve fundamentally *new* problems *not* seen during pre-training, and reveals it as an emergent capability arising from a phase transition... arxiv.org/abs/2306.15063

Excited to share our new work on how pretraining task diversity affects in-context learning.

Want to learn about SGD's implicit bias towards simpler subnetworks generated by permutation symmetry?! Come to our NeurIPS poster session tomorrow morning 10:45 - 12:45 Hall B1+B2 (level 1) #906

Come check out our #NeurIPS2024 spotlight poster on feature learning tomorrow! 📍East Exhibit Hall A-C #2102 📅Thu 12 Dec 4:30 p.m. — 7:30 p.m. PST

Join us at the ML for Multiscale Processes workshop at #ICLR2025 to hear from our three first amazing keynotes: Qianxiao Li blog.nus.edu.sg/qianxiaoli/ Sergei Gukov theory.caltech.edu/~gukov/ Charlotte Bunne aimm.epfl.ch

Proud to be part of the team behind this new open-source SOTA formal math prover! 🚀 Achieving 72.95% on MiniF2F with simple BFS strategy. Our models are trained using expert iteration and DPO, pushing the boundaries of formal theorem proving. 📄 Paper: arxiv.org/abs/2502.03438

The best part of this job is seeing students graduate and launch their careers! Congrats to Feng Chen, Atsushi Yamamura, Tamra Nebabu, Linnie Wharton and Daniel Kunin. They are all going on to top positions across artificial intelligence, medicine, and physics. Proud of you!

AI has always been GPU-first. But on-device AI should be NPU-first. Today we’re launching OmniNeural-4B — the world’s first NPU-aware multimodal model, natively understanding text, images, and audio. And introducing nexaML — a generative AI inference engine that runs models on