A student reached out asking for advice on research directions in optimization, so I wrote a long response with pointers to interesting papers. I thought it'd be worth sharing it here too: 1. Adaptive optimization. There has been a lot going on in the last year, below are some

La Guardia is objectively the best airport

sneak peek of the latest video in the works 👀

if there is model-based reinforcement learning, where is model-cringe RL?

check out Sonica’s paper!! 🔥 we had many fun discussions about it

Several research groups have released papers on the convergence of Muon, mostly looking at either as a Frank-Wolfe method with momentum or a trust region procedure. I find this one to be particularly easy to read.

If the neocortex runs a canonical computation, what is it? The Thousand Brains Theory proposes a sensorimotor algorithm where every column models the world using reference frames 🧠 In the new video, we explore all 6 cortical layers to reveal the biological blueprint:

day 3 of trying out JAX (after years of PyTorch) i’m starting to love it!!

🧵0/7 🚨 Spotlight International Conference on Minority Languages 🚨 Chi-Ning and Hang have been thinking deeply about how feature learning reshapes neural manifolds, and what that tells us about generalization and inductive bias in brains and machines. They put together the thread below, which I’m sharing on

i am not getting a match, am i

When it’s a word pattern matcher, it’s a word pattern matcher. You might think it’s intelligence, but it’s a word pattern matcher. It’s not a belittlement. It’s just what it is.

Brilliant piece by Matt Perich on neural manifolds 🌟 His essay in The Transmitter shows how this approach changes the game in computational neuroscience, reproducing behavioral flexibility within finite neural constraints 🧠 thetransmitter.org/neural-dynamic…

around 3 weeks i disabled GitHub Copilot in my VS Code because I was learning JAX and wanted to check my understanding. I then completely forgot to turn it back on. now i suddenly realized that programming in these past weeks has been much more enjoyable and more efficient...🤔

Nature is healing

Went to an Allbirds store in Cambridge wearing my SpaceX hoodie, and the sales assistant said “I really like your shirt”, so we ended up talking about Starship for a while. He said “I often rewatch the Flight 5”, and I replied “No way, me too!” very good shipping experience

Making a presentation with dark slides and noticed that somehow every single figure from every paper looks much better in dark mode with inverted colors. We need to make white text on black background the default for all publications...

i think about this xkcd comic at least once a week

Never in my life have I thought I would ever use Inkscape over Adobe Illustrator for some reason, yet here we go...

Little linear algebra puzzle: Suppose a,b,c are column vectors. Then (a^T b) is a scalar, and hence (a^T b)c is another column vector. Since column vectors are just tall skinny matrices, and matrix multiplication is associative (AB)C = A(BC) we should expect (a^T b)c = a^T(b c)

we need an International Dendrites Appreciation Day