i have been a big fan of nighttime walks lately. today a bird shit on my hand and then i had to explain duck penises to a couple of strangers. good stuff

> fp8 is 100 tflops faster when the kernel name has "cutlass" in it kms github.com/triton-lang/tr…

Neural networks are grown, not programmed. What does that growth process look like? Like this! This is a small language model (3M) across training, visualised with a new interpretability technique: susceptibilities. We call this handsome critter the rainbow serpent.

new paper :) go read it Right Now

Mom: we have rainbow serpent at home. Rainbow serpent at home: rainbowserpent.dev We recently introduced an approach to interpretability for language models based on susceptibility UMAPs, and it's now available in a webapp for you to try (with some Pythia models too!)

A beautiful quote by Michael Atiyah: In the broad light of day mathematicians check their equations and their proofs, leaving no stone unturned in their search for rigour. But, at night, under the full moon, they dream, they float among the stars and wonder at the miracle of the

i will also buy you coffee, dm me

oh hell yeah

How does training data shape model behavior? Well, it’s complicated… 1/10

Training Data Attribution (TDA) should account for learning dynamics! The same data can influence model behavior in dramatically different ways at different time points of training. We call for a shift towards stagewise data attribution and the study of influence dynamics. 1/11

forgive me bryan johnson

mom i made it into the nyt

Soon, most thoughts on Earth will be carried by tokens. Many beautiful; some consequential. Understanding this rising sea of intelligence is a major scientific problem, and is the aim of interpretability. Our new interp results on susceptibilities for Pythia-1.4B: 🧵