My first serious take on short story. Hope you enjoy! link.medium.com/Erg6VKUauqb

I have struggled to find intuition in the derivation of policy gradients and connection between RL and supervised ML. Yesterday I spent sometime to walk through those issues and figured out a way to understand both on a deeper level: medium.com/@tonychenxyz/u…

For a reader-friendly, intuitive resource to help you bettter understand policy gradient and how it works, don't miss Tony Chen @ ICML 24's in-depth primer. buff.ly/42F5nI8

Too much technology is built to impress the people who build technology. Technology should enable people to build and impress.

Multimodal pre-trained models, such as CLIP, are popular for zero-shot classification due to their open-vocabulary flexibility and high performance, but how would you classify images that don’t have obvious names using CLIP?

Excited to share that SelfIE is accepted to ICML 2024! Project page: selfie.cs.columbia.edu Paper: arxiv.org/abs/2403.10949 Code: github.com/tonychenxyz/se…

Come check out our poster on interpreting hidden layer embeddings of vision transformers with text explanations at ICLR at booth #97 today at 10:45am! Paper: openreview.net/pdf?id=5iENGLE…

🚨 New paper! 🚨 We solve lots of tasks posed in words by thinking visually. Can LLMs? Not in text, but we can unlock this ability with images! Introducing whiteboard-of-thought, enabling MLLMs to express intermediate reasoning as images via code! 🔗 huggingface.co/papers/2406.14… 🧵

How can a visuomotor policy learn from internet videos? We introduce Dreamitate, where a robot uses a fine-tuned video diffusion model to dream the future (top) and imitate the dream to accomplish a task (bottom). website: dreamitate.cs.columbia.edu paper: arxiv.org/abs/2406.16862

Arrived in Vienna today for #ICML2024! Come check out our poster on zero-shot open-ended interpretation for LLM hidden embeddings (and how we made llama think Molotov cocktail is a drink)! Time: Tue 23 Jul 1:30 p.m. CEST — 3 p.m. CEST Location: Hall C 4-9 #800