In the late 1960s top airplane speeds were increasing dramatically. People assumed the trend would continue. Pan Am was pre-booking flights to the moon. But it turned out the trend was about to fall off a cliff. I think it's the same thing with AI scaling — it's going to run

'every exponential (system) is a sigmoid in disguise' is such a profound observation

'values are only interesting to the extent that the opposite of your value is also viable'

This MIT research as well as anthropics monosemantic paper discovered that LLMs converge to same concept representations across models and modalities. wow! reminds us of word2vec analogies. I believe this is the root of observed intelligence in LLMs

200x compression is mathematically impossible for this data. Given the specs of Neuralink's chip, you cannot do better than 5.3x compression. Here's why 🧵 (1/10)

My google exp reinforced a few learnings for me: (1) consumers buy products; enterprises buy platforms. (2) distribution advantages overtake product / tech advantages and (3) companies that reach PMF & then under-invest in S&M risk staying niche players or worse: get taken down.

Elon Musk open-sourced all of Tesla's patents for other car companies to copy. Wall Street called it 'dumb' and thought Tesla will be crushed. Elon was right. Everyone was wrong. Every entrepreneur needs to understand why he did it and why it worked:🧵

I'm shocked. Just shocked that continually averaging a data corpus without exogenous inputs results in degraded information quality. Doomer ouroboros arguments were always silly. But they get demonstrably sillier as the research continues. nature.com/articles/s4158…

If you set reasonable milestones, you will only make reasonable progress. unreasonable people drive to make unreasonable progress.

“Experience is what you get when you didn't get what you wanted.” “When you're screwing up and nobody's saying anything to you anymore, that means they gave up.” Video: youtube.com/watch?v=ji5_Mq… Book: amazon.com/Last-Lecture-R… 2/4

What’s the difference between reason, intelligence and understanding? Reason is all methods of criticism that improve ideas. Intelligence is how fast we create ideas that solve problems. Understanding is the ability to address criticisms of an idea. Rationality is seeking to

The benefits of caloric restriction for primate lifespans (including our own!) are probably overrated. We have two rhesus monkey studies, and they do not support big benefits🧵 First, take a look at this diagram:

Deepmind chess paper showed that deterministic software can be mapped to neural nets arxiv.org/abs/2402.04494 This allows us to start from existing knowledge, but design the system to self-improve it through experience

The (true) story of development and inspiration behind the "attention" operator, the one in "Attention is All you Need" that introduced the Transformer. From personal email correspondence with the author 🇺🇦 Dzmitry Bahdanau @ NeurIPS ~2 years ago, published here and now (with permission) following

Great to see meaningful usecases of reasoning models

Came across this neat tool with viz to dig into llm internals: bbycroft.net/llm

one way to think about ai: its humanity's collective brainpower over time i think we'll get experts and fragments to this central brain, but directionally its "collective intelligence"

FROM MAGA TO CHINA Here are four things MAGA is getting wrong, and why it's handing over the world to China. (1) First, MAGA correctly understands that America’s economic position is in decline but thinks this is due to economic competition itself, rather than lack of

TL;DR: We built a transformer-based payments foundation model. It works. For years, Stripe has been using machine learning models trained on discrete features (BIN, zip, payment method, etc.) to improve our products for users. And these feature-by-feature efforts have worked