New preprint: “Removing Biases from Molecular Representations via Information Maximization” with Tommi Jaakkola, Dr. Caroline Uhler, Sharut Gupta Tackling batch effects in multimodal molecular representation learning with high-content drug screening data. arxiv.org/abs/2312.00718

Presenting "Compositional Sculpting of Iterative Generative Processes" @ #NeurlPS2023 today! Poster #712 5pm-7pm Come by to chat about mathematical operations and algorithms for the composition of generative models. Links: neurips.cc/virtual/2023/p… timgaripov.github.io/static/files/2…

Extremely excited to have this work out, the first paper from the Superalignment team! We study how large models can generalize from supervision of much weaker models. x.com/OpenAI/status/…

InfoCORE has been accepted to #ICLR2024! Explore our generic framework designed to mitigate confounding factors in multimodal representation learning, proving effective on real-world molecule datasets.😎 Paper: arxiv.org/abs/2312.00718 Code: github.com/uhlerlab/InfoC…

Weight averaging and model merging for LLMs seem to be the most interesting themes in 2024 so far. What are the benefits? Combining multiple models (or checkpoints) into a single one can improve training convergence, overall performance, and also robustness. I will probably do

Combining discrete and continuous data is an important capability for generative models. To address this for protein design, we introduce Multiflow, a generative model for structure and sequence generation. Preprint: arxiv.org/abs/2402.04997 Code: github.com/jasonkyuyim/mu… 1/8

I moved to SF and joined OpenAI.

Model merging is a popular research topic with applications to LLM alignment and specialization. But, did you know this technique has been studied since the 90s? Here’s a brief timeline… (Stage 0) Original work on model merging dates back to the 90s [1], where authors showed

Earlier this week, I published a model merging research timeline, and the feedback led me to a bunch of really interesting / useful papers. Here's a summary of my favorites... Mode connectivity, which has deep relations to the inner workings of model merging, was originally

Today, I’m excited to share with you all the fruit of our effort at OpenAI to create AI models capable of truly general reasoning: OpenAI's new o1 model series! (aka 🍓) Let me explain 🧵 1/

o1 loves solving good crosswords and make interactive artifacts in retro scifi vibes for that also how cute the reasoning summaries are 😍 "hm, i'm thinking!"

Training of o1 has been conducted by an amazing team. By far the best team I've seen in my life. If you are an outstanding researcher or engineer who wants to study frontiers of reinforcement learning for reasoning, we still have a few roles we want to hire for o2 😉

For the first time we are fundamentally changing how humans can collaborate with ChatGPT since it launched two years ago. We’re introducing canvas, a new interface for working with ChatGPT on writing and coding projects that go beyond simple chat. Product and model features:

The partition of science and human knowledge into fields is a social construct.

Transformers can be trained to solve a 132-years old open problem: discovering global Lyapunov functions. New paper on Arxiv (accepted in NeurIPS 2024), with Alberto Alfarano and Amaury Hayat arxiv.org/abs/2410.08304 1/8

I'm looking for two more strong REs/SWEs to work on the future of o1 at OpenAI 🤖 If you're excited about reasoning and AGI, fill out this form: jobs.ashbyhq.com/openai/form/oa… (also, I'll be at NeurIPS if you want to chat about🍓)

I am super lucky to have worked with Pavel. In addition to being a terrific researcher, Pavel supported me personally through the ups and downs of my life. Apply to Pavel's group if you want to engage in a rare mix of impactful empirical work in AI and foundational ML research.

Back to the top 🚀🚀🚀🚀🚀

Alex, congratulations to you and the team! Amazing work!