San Francisco is so back.

"Toward General Virtual Agents" I recently gave a talk at MIT. I argued that we should use tools from reinforcement learning and search to improve the capability and alignment of LLM agents. Slides: drive.google.com/file/d/1kDvmrm… Video:

AlphaCode (powered by Gemini) is now roughly as capable as I am ("entry-level Division 1") on CodeForces -- a feat I did not expect to see this soon!

Whoever tells you “we understand deep learning” just show them this. Fractals of the loss landscape as a function of hyperparameters even for small two layers nets. Incredible

Since AlexNet/ImageNet moment in October 2012, stock multiples: nvidia: ~250x amd: ~80x intel: ~2x

"We do these things not because they are easy, but because we thought they were going to be easy" is a surprisingly profound quote. When I ask people who've pulled off remarkable things, it's interesting how many confirm that they wouldn't have started if they'd know how long and

Meta announces Aria Everyday Activities Dataset present Aria Everyday Activities (AEA) Dataset, an egocentric multimodal open dataset recorded using Project Aria glasses. AEA contains 143 daily activity sequences recorded by multiple wearers in five geographically diverse indoor

SOTA AI for games like poker & Hanabi rely on search methods that don’t scale to games w/ large amounts of hidden information. In our ICLR paper, we introduce simple search methods that scale to large games & get SOTA for Hanabi w/ 100x less compute. 1/N arxiv.org/abs/2304.13138

Happy to share our work on reproducing RLHF scaling behaviors in OpenAI's work in summarizing from feedback. We built an RLHF pipeline from scratch and enumerated over 20+ implementation details 🚀 Fun collab with Michael Noukhovitch @NeurIPS 2024, Arian Hosseini @ NeurIPS, Kashif Rasul, wang, and Lewis Tunstall 📜

Excited to share Penzai, a JAX research toolkit from Google DeepMind for building, editing, and visualizing neural networks! Penzai makes it easy to see model internals and lets you inject custom logic anywhere. Check it out on GitHub: github.com/google-deepmin…

And now we know reddit.com/r/reinforcemen…

Deep RL is finally back out of the Trough of Disillusionment

I gave my first guest lecture today in a grad course on LLMs as an (soon-to-be) adjunct prof at McGill. Putting the slides here, maybe useful to some folks ;) drive.google.com/file/d/1komQ7s…

Today feels like the future. "Her" arrives on my phone (from OpenAI), AR glasses prototypes are getting good (Meta), and humanity can produce this with a drone swarm. Let alone CRISPR, commercial space, etc. Not sure if it is good, but it's a wild ride. x.com/ShenzhenPages/…

"In the Beginning, ML was RL". Andrew Barto gave RLC 2024 an amazing overview of the intertwined history of ML and RL (Link below)

Tech that looks like it’s from 2001 >

I know it seems like Amazon is eating Anthropic, with the funding and preferred compute partnership. However, I think the sub-title here of "deep technical collaboration" on "directly interfac[ing] with Trainium silicon" to improve the story of AWS chips could actually be immense

How to know when you’re hitting a wall*: watch for when the benchmark rigor police start coming out in force. It happens in every epoch of AI research. Whenever the gains of a paradigm slow, the rigor police reemerge in the vain hope that the objective paradox can be thwarted.

Very happy to hear that GANs are getting the test of time award at NeurIPS 2024. The NeurIPS test of time awards are given to papers which have stood the test of the time for a decade. I took some time to reminisce how GANs came about and how AI has evolve in the last decade.

In my 2019 AI-GA paper I proposed a neural net world model as a "Darwin Complete" environment search space that could produce any possible environment for open-ended learning. It felt like a flight of fancy. I knew rationally it was possible eventually, but emotionally it felt