On different machines, the following code returns slightly different results, even with the same torch library. It turns out that the low-level math acceleration library is different for different CPUs, and that alone can cause tiny difference that accumulates.

It took decades—and many of us—to (re)discover what Hinton described as coarse coding 40 years ago. He saw it clearly through simple 2D dot examples, not noisy, overcomplicated methods. He shaped the future by seeing it before anyone else.

Interpretation in LLM is illusion. Causality is coincidence.

I wrote a blog post that recreate the invention process of VAE, Diffusion, and Flow Matching in a science fiction style. It doesn't have difficult mathematics, only intuition, logic, and reasonable imagination. peiguo.me/posts/autoenco…

I'm reading On Intelligence by Jeff Hawkins, and I have this mind-numbing feeling, like he somehow traveled back from the present to write this book.

I wish I had watched Chelsea Finn's 'how to do research' talk a decade ago instead of focusing on so many online resource on 'how to publish a paper'. youtu.be/FacJ_1tTSx4?li…

I hope I have watched this ICML DL Workshop 2015 - Panel Discussion 15 years ago. youtube.com/watch?v=EiStan…

Choosing whom to follow is an important part of one's research taste, if not the most important. If I can choose again, I would learn neural nets from Hinton himself 10 years ago instead of wasting my time studying from so much misleading online materials.

I've known E=mc^2 for long but never read Einstein's original paper until recently. It only has 3 pages and deduces the equation by looking at an imaginary body emitting lights from two coordinate systems. Simple math, great intuition. sites.pitt.edu/~jdnorton/teac…

Hopfield networks laid the groundwork for AI research, representing a rare and fruitful marriage between neuroscience and statistical physics.

Most AI researchers have to be both theorist and experimentalist—coming up with ideas and implementing them. Now that I can delegate the coding to Claude, I get to be a pure theorist for the first time.

Many times when I found something interesting, like Johnson–Lindenstrauss lemma, I can find a nice blog from Su Jianlin. spaces.ac.cn/archives/8679

Both Demis Hassabis and Dario Amodei studied neuroscience during their PhDs — that fact alone seems to carry some significance.

Annus Mirabilis!!

Only after this video have I realized computer vision has learned so much from human vision, and there's no reason to believe this trend won't continue. youtube.com/watch?v=Gv6Edl…

New blogpost exploring interpretable features from random networks. In it I explore the cause of this phenomenon: is it a property of the network or does it reveal some limitation of the interpreting tools? The answer is both. peiguo.me/posts/random-e…