Want to do research in ML for science in NYC and develop the next generation of foundation models for scientific data? Apply here to be a postdoc at PolymathicAI Simons Foundation : forms.gle/JbdWA3VczH3Xfg…

We are excited to share three major updates today on our path to mathematical superintelligence 🦾 1. A new state-of-the-art of 90% on the MiniF2F benchmark. This beats our previously announced 83% from under a month ago.

Can Transformers extrapolate from short training sequences to long ones? Our new work shows that they display surprising “length generalization” capabilities on many algorithmic tasks: addition, multiplication, and even in-context simulation of SGD! arxiv.org/abs/2407.03310

Actively learning from weak information is a key problem in machine learning. Presented at COLT last week, our paper captures core aspects of active weakly supervised learning, showing how to combine information-theoretical and bandit likes methods. arxiv.org/abs/2402.13079

Today, we're releasing LAB-Bench, a set of >2000 evaluations for language models and agents on scientific research tasks in biology. Public models underperform PhD/postdoc-level humans on nearly all tasks. Claude 3.5 Sonnet is the clear frontrunner atm, but long way to go. 1/

mistral.ai/news/mathstral/ mistral.ai/news/codestral…

Join us at PolymathicAI at Simons Foundation's Flatiron Institute to create AI models for science! Opportunities available for postdoctoral candidates! Google form below: forms.gle/JbdWA3VczH3Xfg…

In Vienna for ICML 2024! I'll be presenting our paper on how transformers learn causal structure (like the induction head) via GD at the Wednesday 1:30pm poster session (Hall C, poster #1600). Excited to chat with all who are around! x.com/EshaanNichani/…

We have a spotlight paper at #ICML2024 on batch and match: black box variational inference with a score-based divergence. Our poster is Thursday 11:30a Hall C 4–9 #1307. 1/n

At ICML? Come check out the Illusion of State poster🪄♟️ Wednesday 11:30 Hall C 4-9 #1013 icml.cc/virtual/2024/p…

New method for VI (Variational Inference) and a new optimization trick! Let me tell you about our new (spotlight) paper at ICML (Thu 11:30am Hall C 4-9 #1307). arxiv.org/pdf/2402.14758 The story involves a new divergence and a (potentially) new proximal point update. 1/n

Training LLMs involves learning associations. We study training dynamics in a simple model that yields useful insight on the role of token interference and imbalance. Go talk to Vivien Cabannes and Berfin Simsek @ ICML at our #ICML poster #1114 tmrw 11:30! Paper: arxiv.org/abs/2402.18724

Want to build powerful AI for Science? We at PolymathicAI are building foundation models for science 🔥 at Flatiron Institute Join us ( Alberto Bietti Miles Cranmer Kyunghyun Cho Michael Eickenberg Siavash Golkar Mariel Pettee, Francois Lanusse, Mike McCabe, Tom Hehir, Rudy Morel,

Now that I have started using twitter somewhat regularly, let me take a minute to advertise the RL theory lecture notes I have been developing with Sasha Rakhlin: arxiv.org/abs/2312.16730

Some exciting PolymathicAI news... We're expanding!! New Research Software Engineer positions opening in Cambridge UK, NYC, and remote. Come build generalist foundation models for science with us! Please indicate your interest on the form here: docs.google.com/forms/d/e/1FAI…

SOTA models often use bidirectional transformers for non-NLP tasks but did you know causal transformers can outperform them even on tasks without a causal structure? Our recent work shows causal transformers learn circuits bidirectional ones can't, leading to better performance!

The application deadline is September 1st!

I'm looking for more reviewers (especially in optimization theory) for Transactions on Machine Learning Research, please reach out with your email (you can send it to me in DM) if you're interested in becoming a reviewer! TMLR has the best system I know: 1. You don't get assigned more than 1 submission at a

magnet:?xt=urn:btih:7278e625de2b1da598b23954c13933047126238a&dn=pixtral-12b-240910&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=udp%3A%2F%2Fopen.demonii.com%3A1337%2Fannounce&tr=http%3A%2F%2Ftracker.ipv6tracker.org%3A80%2Fannounce

Excited to announce that ML-NYC is back this semester. Our first speaker is Bin Yu from UC Berkeley on Monday Sept 23rd 4pm. Register here: eventbrite.com/e/ml-nyc-speak…