New paper - Transformers, but without normalization layers (1/n)

The way interpretability unlocks innovation is not new. Consider biology after Nirenberg cracked the genetic code in 1962. Solving "biological interpretability", decoding the mysterious information locked inside cells, created an explosion of innovation in modern medicine.

Looks very useful for interp practioners!

AI models are *not* solving problems the way we think using Docent, we find that Claude solves *broken* eval tasks - memorizing answers & hallucinating them! details in 🧵 we really need to look at our data harder, and it's time to rethink how we do evals...

Excited to share that I've joined @FirstRound to invest in exceptional founders at the earliest stages of their journeys. 🚀 First Round backed some of my close friends last year (special thanks to Adit for the introduction and Caleb Harris for the support), and it's been a

Great news! Undergraduates from families earning below $200,000 can attend MIT tuition-free. This initiative will make higher education more accessible to 80% of American families, ensuring that talented students can pursue their dreams without financial barriers.

We have a small gift for the open-source community: RolmOCR, a new OCR model for complex document processing! We at Reducto trained a Qwen2.5-VL-7B model (by Qwen ) using the amazing olmOCR dataset by Ai2 earlier this year. As a result, we have RolmOCR - a

Bio foundation models are great design and engg tools. But can they help decode the fundamental principles of life? We harnessed a single-cell FM for decoding the long-debated relationship between genome arch. and gene coregulation. It all started with an idle curiosity. 1/

Our paper "Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?" will be presented at #ICLR2025! It's also the first paper of my first PhD student — congrats @maximemeloux! 🎉 blog post: melouxm.github.io/MI-identifiabi… A short thread 🧵

I’m at #ICLR2025 the next few days! I’m making lots of time in my schedule to talk to researchers who might someday want to start a company. If that’s you, sign up for a YC office hours with me here. events.ycombinator.com/yc-oh-iclr

Biology’s lack of data is holding back its AI boom. Epoch’s latest report shows explosive growth in biological model training data size from 2017–2021 (9.7×/year), but a (2.1x/year) plateau since. AI models for biology are ready to transform science if the data can keep up.

The YC Summer 25 application deadline is May 13th! No better time to found & build tech that'll shape the next decade. To pay it forward, I'm happy to review applications & mock interviews so reach out! ycombinator.com/apply

Kaiwan Turel, awzf , and I were researching long horizon reasoning (with Jacob Andreas). We found existing benchmarks’ hard problems often featured tricky puzzles, not tests of system understanding. So we made Breakpoint: a SWE benchmark designed to disambiguate this capability.

2/3 companies I mocked this batch got in! Feeling proud. Highest leverage 15 min every time.