huge fan

what's missing with jax?

lecun knows that's not gonna be it. off-ramp has a very specific meaning and it's not "the destination mimus a couple of steps". unless ur definition of LLMs really just boils down to big neural net that predicts the future as an objective, but it's prob not.

too much message passing

x.com/i/article/1941…

Feynman on beauty of the flower: I have a friend who’s an artist and has sometimes taken a view which I don’t agree with very well. He’ll hold up a flower and say “look how beautiful it is,” and I’ll agree. Then he says “I as an artist can see how beautiful this is but you as a

funny how we humans tend to overestimate our own engineering while underestimating our own... engineering

don't take this lightly. i've come to see love as a fragmentation of the soul, not too dissimilar to rowling's idea of a horcrux. every time u fall in love, and i mean actual love, u slowly but inevitably and continuously break pieces of urself and offer them to ur significant

i'm pretty sure this could be used as a foundation to explain behavior of some of the new verifier-free RL methods, as both entropy minimization and self-consistency boil down to minimizing an energy function conditioned on the sequence (in one case it's entropy and in the other

it always surprises me the degree to which this take is popular among top ML researchers. for a field founded on the ambition of reproducing human intelligence, u'd at least assume a non-reductive understanding of the brain as a prerequisite. a 100T param GPT would not be

"you say opus 3 is close to aligned – what's the negative space here, what makes it misaligned?" I've been thinking more about how to answer this because it's a very good question, and in particular about the distinction between issues that seem naturally resolved if Opus 3 is

the problem is real

> Unfortunately, this means that I think the field has made almost no progress on actually figuring out what credit assignment algorithm the brain uses and has instead just been exploring the sterile arcana of weird approximations to backprop. ~ Beren Millidge has a great blog

nice thesis Michael Timothy Bennett

nascent but interesting...