Don't let low-order statistics fool you

This blog is so excellent that I couldn't help sending a tweet to compliment it!

The correct way to tie a scarf

quite impressive

In the LLM-science discussion, I see a common misconception that science is a thing you do and that writing about it is separate and can be automated. I’ve written over 300 scientific papers and can assure you that science writing can’t be separated from science doing. Why? 1/18

Three years ago today, the project that eventually became NeRF started working (positional encoding was the missing piece that got us from "hmm" to "wow"). Here's a snippet of that email thread between Matt Tancik, Pratul Srinivasan, Ben Mildenhall, and me. Happy birthday NeRF!

A poster by Larry Wasserman to put on your refrigerator.

lol

I agree with Jim Fan. Life, with all its midblowing structure, is about creating order in a universe of increasing disorder, see e.g. newscientist.com/article/232382… for an easy intro. Like a cell, a neural network during training takes energy to minimise disorder, that is to predict

😂

Historic Vids I'm crying, it's way too cute 🥰

The #NobelPrizeinPhysics2024 for Hopfield & Hinton rewards plagiarism and incorrect attribution in computer science. It's mostly about Amari's "Hopfield network" and the "Boltzmann Machine." 1. The Lenz-Ising recurrent architecture with neuron-like elements was published in

Classic Japanese ad for batteries from 40 years ago. Kawaii is all you need.

lol

Finally had a chance to listen through this pod with Sutton, which was interesting and amusing. As background, Sutton's "The Bitter Lesson" has become a bit of biblical text in frontier LLM circles. Researchers routinely talk about and ask whether this or that approach or idea

The Andrej Karpathy interview 0:00:00 – AGI is still a decade away 0:30:33 – LLM cognitive deficits 0:40:53 – RL is terrible 0:50:26 – How do humans learn? 1:07:13 – AGI will blend into 2% GDP growth 1:18:24 – ASI 1:33:38 – Evolution of intelligence & culture 1:43:43 - Why self

lol I love the pretraining idea!