"It's not hard for Zuck to poach OpenAI talent, not just because he has the money, but because open-source AI is fulfilling the original OpenAI mission." This is brutally true.

Revisit FlashAttention: I often appreciate revisiting some classic work developed in AI, as it helped me learn many insights. Among them, FlashAttention is a nice example of how fundamental knowledge might drive AI breakthroughs. It optimizes attention by combining: 1.

🎉 Just arrived at #ICML2025! We’ll be presenting our DPLM-2.1 poster on Tuesday. Come chat with me and Zaixiang Zheng about protein modeling and the next generation of diffusion protein language models (DPLM). 📍 Location: West Exhibition Hall B2-B3, Poster #W-115 🕓 Time:

🎉 DPLM-2.1 at #ICML2025! It's happening today (7/15). We’ll be presenting our DPLM-2.1 poster. Come chat with me and Zaixiang Zheng about protein modeling and the next generation of diffusion protein language models (DPLM). 📍 Location: West Exhibition Hall B2-B3, Poster

🎉 DPLM-2.1 at #ICML2025! It's happening now! 📍 Location: West Exhibition Hall B2-B3, Poster #W-115 🕓 Time: Tuesday, July 15, 4:30–7:00 p.m. PDT Come chat with our team about protein modeling and the next generation of diffusion protein language models (DPLM). Learng more

I always believed that talents from CMU Robotics would go on to build something remarkable. It's inspiring to see Skild AI pushing the boundaries of robotics. From a technical standpoint, their robots' ability to generalize over diverse real-world scenarios is particularly

Our paper is now featured on the official ByteDance SEED publications page. “Elucidating the Design Space of Multimodal Protein Language Models” (ICML 2025 Spotlight) explores the intersection of structure and sequence for multimodal protein foundation models. Check it out

Great article. I've always wondered how language models generate text when the context length exceeds the training context window. You could naively increase the kv cache size, but the performance would degrade dramatically. Attention sinks shows that the first few tokens (four