EvolutionaryScale.ai : an AI-for-proteomics startup that just came out of stealth. They are announcing ESM3 a 98B-paramter generative LLM for "programming biology." Using ESM3 and a simulated evolutionary process, they have produced a new type GFP (Green Fluorescent Protein)

🧵 Check out what I've been working on at EvolutionaryScale 🧵 We trained a bidirectional masked generative model on protein sequence, structure and function! From 1.4B to 98B parameters, our models are so good that beating ESMFold was just a footnote 🤯 👇👇👇

🧵Today is an exciting day! We have just announced ESM3, the design of a new green fluorescent protein, and the launch of our company. Here's a short thread about the scaling and infrastructure work involved in making ESM3 happen! 🧵

I'm so excited to share that I've been at EvolutionaryScale, it feels like a dream come true! As one of our first projects, we took on designing a new green fluorescent protein with ESM3, our frontier generative language model for programming biology

I’m so excited to share what we’ve been working on EvolutionaryScale. ESM3 is a multimodal generative masked language model for programming biology. Here’s a short thread on the architecture behind ESM3. 🧵 x.com/alexrives/stat…

Working on ESM3 has been the most challenging and the most rewarding part of my career. I am incredibly proud of the team we have built - y’all make it so fun to come in to work each day.

Read Nicholas Sofroniew’s deep dive on esmGFP here!

esm3 is out it is a frontier multimodal generative language model that reasons over the sequence, structure, and function of proteins. the model can follow complex prompts combining its modalities and is highly responsive to biological alignment. interesting tid bits:

Weekend project: Comparing ESM3 from EvolutionaryScale to ESM2 and inv_cov. The ultimate test of a protein language models is how well the pairwise dependencies it learns correlate to structure. (1/8)

biorxiv.org/content/10.110… We're live!

After we trained ESM3, we were excited to see it could respond to complex prompts, which greatly improved with scale. While scaling has been driving advances across AI—in biology it has been largely underestimated. Article by Ewen Callaway for nature: nature.com/articles/d4158…

Great application of ESM-2: Accurately resolving the evolutionary trajectories of SARS-CoV-2, all starting from a single sequence! In the near-term future, viral genomic surveillance could look very different...

The Machine Learning in Structural Biology workshop will be back at #NeurIPS for its 5th edition in December! Stay tuned as we will soon share details about the stellar lineup of speakers 🌟, the call for papers 🧬, and a very exciting innovation for this year's workshop 🥁🚀!!

As promised, the next 3 weeks of speakers for Machine learning for protein engineering seminar : 07/23, Roshan Rao with ESM3 07/30, Jason Yang with CARE 08/06, Alex Tong with Sequence-Augmented SE(3)-Flow Matching Visit our website to sign up for slack and our g-group newsletter!

The ESM3 package was updated on pip. We fixed an issue with iterative structure prediction. Expect to see major improvements after doing: pip install -U esm

To provide real value, generative molecular methods really do need a litany of features and control modalities which each deserve their own models and attention. At Terray we use latent diffusion to build up a suite of models that build towards that. biorxiv.org/content/10.110…