The moat of software AI agents is not the thin wrapper layer (Devin, SWE-Agent), but the underlying LLM. Instead of benchmarking the wrapper, I think SWE-Bench is excellent for evaluating coding LLMs instead:

Hold the agent layer fixed and vary only the LLM backend. Provide all…

Math talking to bare metal in the purest way. Andrej Karpathy makes AI education not only accessible, but also elegant. I'm reading through the code like a work of art.

The legendary class created by Fei-Fei Li & Andrej Karpathy that introduced deep learning to a generation of students. Proud to be a TA alumnus for CS231n! I used to write the Google Cloud tutorial on how to set up GPU instances and run experiments ;)

Better manual design of the command line tools for GPT-4 is all you need to get 12.3% on SWEBench. There is no magic, no model breakthrough, no justification for the extreme hype.

When GPT-5 comes, instruction following, tool use, and long context will surely be far better. None…

This sakura video has no more complexity than 262 characters, implemented as shader code that *generates* pixels. A text2video model that achieves maximal possible compression will be able to recover this program approximately in its weights, synthesized through denoising and…

Novelty is so overrated. It's an example of misaligned objective: if reviewers look for novelty, you shape your research and efforts towards that, while devaluing things that actually matter.

I used to review CVPR papers, but stopped wasting time on so many mind-numbing papers…

Many of us are so deeply immersed in technical details that we forget what purpose AI is serving. AI is meant for the benefit of all humanity from all walks of life. Fei-Fei Li's fireside chat is a breath of fresh air. She offers a warm, human touch on cold, calculating machines.…

Foundation Agent: a roadmap to build generally capable embodied AI that acts skillfully across many worlds, virtual or real.

Project GR00T, the Humanoid robot foundation model, is a cornerstone for Foundation Agent. It's the North Star, the next grand challenge in our quest for…

Thank you Charles for the wonderful summary! Hosting Percy Liang for fireside chat was so easy: he’s got way too many great works to celebrate. We only ran out of time, but never ran out of deep insights on foundation models!

Yesterday, during the GTC Keynote session, Jensen announced NVIDIA's new moonshot project, GR00T! This ambitious endeavor aims to create a general-purpose foundation model for humanoid robots, necessitating advances in and the convergence of various fields such as simulation,…

2024 is the Year of Humanoid. There’s no robot hardware more general-purpose. We are all in.