YOW-SFO should be a direct flight

This mini-review is based on Hugo Cui's PhD thesis: arxiv.org/abs/2409.13904 . My advice to him was: "Write something you would have loved to have when you started your PhD!" He did an outstanding job introducing the rich methods he developed. Enjoy and share widely!

I remember when I started working on ML many years ago and people told me it wasn't physics

John Hopfield has a nice article in the annual reviews of condensed matter physics. It starts off with a discussion of what physics is, which I think is totally on point.

Today the Nobel committee has confirmed my long-held definition of Physics: Things I find interesting.

Let me share a lecture from my ML for Physics course at EPFL, where I derive the Boltzmann machine from the maximum entropy principle and use it to motivate the attention mechanism and a simple transformer architecture. Enjoy! youtu.be/dwMhw2X8_TU youtu.be/od_XiaCJzV0

unpopular opinion: ML conferences should charge $100 per submission. For accepted papers this would count towards the registration fee of the attending author, so it's free. Extra funds collected could be used eg. for replication studies or other improvement to the review process

Are you doing research in AI and looking for a full-time position in industry? Come work with me at Block. (we are a remote friendly company :) block.xyz/careers/jobs/4…

LLMs are powerful sequence modeling tools! They not only can generate language, but also actions for playing video games, or numerical values for forecasting time series. Can we help LLMs better model these continuous "tokens"? Our answer: Fourier series! Let me explain… 🧵(1/n)

Congratulations to COP 475 for continuing to under deliver! Don't worry about it, I'm sure you will get it right next time!!! cbc.ca/news/world/cop…

I just assumed Trudeau was sticking around until he could go to the Eras Tour.

How do you make people ignore a $60 billion dollar hole? Fire someone famous.

Headlines about inflation are getting annoying. Why do people expect prices to go DOWN, when inflation is still a positive number? cbc.ca/news/business/…

🤣🤣

Pictured: me giving up on language model research in 2011.

ChatGPT model naming conventions are terrible. The interface has no less than 7 different model options to choose from...

Anyone else having audio issues with TSN right now? It’s all static #SuperBowlLIX

Hopefully KC sobers up by the 2nd half

So, this is it then? No more OOMs of scaling, no more speculation about whether GPT-5, 6 or 7 would be AGI. The performance wasn't even good enough to brand it as GPT-5. The diminishing returns finally kicked in and this is as far as we'll get just from scaling pretraining.