LeCun's "5 best ideas 2012-22” are mostly from my lab, and older: 1 Self-supervised 1991 RNN stack; 2 ResNet = open-gated 2015 Highway Net; 3&4 Key/Value-based fast weights 1991; 5 Transformers with linearized self-attention 1991. (Also GAN 1990.) Details: people.idsia.ch/~juergen/lecun…

More generally, the narrative that we're in decline (in tech or otherwise) because of "wokeness" (a term used exclusively by a certain kind of people to designate everything they don't like about social progress, the way they might have used "hippies" in the 1960s) is nonsense.

GPT-4 will have the same number of parameters as GPT-3, but these parameters will be kiki instead of bouba

Julia 🚀🚀🚀🚀🚀🚀🚀🚀🚀🚀 Python 🐌🐌🐌🐌🐌🐌🐌🐌🐌🐌 Rust🐌 #python #programming #backend

Please remind me how we are moving forward. In this video, a machine from the year ~2000 (600MHz, 128MB RAM, spinning-rust hard disk) running Windows NT 3.51. Note how incredibly snappy opening apps is. 👇

Ok sure it's cool to see Andrej Karpathy 's llama2.c but what about llama2 in holy-c

Python is a great language if you don't want to use for loops

When can I buy TPUs?

Our ability to do things is dying a death of 1000 cuts. It's nice to talk about bottlenecks and silver bullets, but my hypothesis is that stagnation is the accumulation of many small frictions. My plan is to continuously update this thread with examples as they pop up.

Not having any sex lately has helped to understand LLVM better

That's how you have to talk to chatGPT

That's it I'm voting for Trump

🫃

“2025 is gonna be my year” August: