Thanks Weixin Liang We all enjoyed reading the paper! And we appreciate your paper for helping the community gain a deeper understanding of the modality gap 🥰

Honored that nature has highlighted our work again in their latest piece examining #ChatGPT's transformative impact on scientific research and academia over the past two years. h/t nature nature.com/articles/d4158…

I’m open to academia & industry in 2025. My work in #XR 🥽 + #HCI 👩‍💻 enables low-friction XR experience thru #EmbodiedInteraction, unlocking potential for all -- tech-savvy or not 🌍 Design+Science+Engineering. Let's shape the future of spatial computing ✨ RT appreciated! (1/8)

🤔 Why are VLMs (even GPT-4V) worse at image classification than CLIP, despite using CLIP as their vision encoder? Presenting VLMClassifier at #NeurIPS2024: ⏰ Dec 11 (Wed), 11:00-14:00 📍 East Hall #3710 Key findings: 1️⃣ VLMs dramatically underperform CLIP (>20% gap) 2️⃣ After

Introducing 𝐋𝐥𝐚𝐦𝐚𝐅𝐮𝐬𝐢𝐨𝐧: empowering Llama 🦙 with diffusion 🎨 to understand and generate text and images in arbitrary sequences. ✨ Building upon Transfusion, our recipe fully preserves Llama’s language performance while unlocking its multimodal understanding and

🔍 Vision language models are getting better - but how do we evaluate them reliably? Introducing AutoConverter: transforming open-ended VQA into challenging multiple-choice questions! Key findings: 1️⃣ Current open-ended VQA eval methods are flawed: rule-based metrics correlate

🔍 ML library development is crucial but requires expertise in ML algorithms & architecture-specific programming languages (ASPLs). 🤖 LLM agents can enable better automation. We propose an adaptive self-improvement agentic system for generating ML libraries in STeP—a

📢 Can LLMs program themselves to run faster? 🏃⏱️ LLM self-taught to code for next-gen AI hardware! arxiv.org/abs/2502.02534 1/ Programming AI accelerators is a major bottleneck in ML. Our self-improving LLM agent learns to write optimized code for new hardware, achieving 3.9x

🚀 Want 2x faster pretraining for your multi-modal LLM? 🧵 Following up on Mixture-of-Transformers (MoT), we're excited to share Mixture-of-Mamba (MoM)! arxiv.org/abs/2501.16295 🔥 Why it matters: MoM applies modality-aware sparsity across image, text, and speech—making

We introduce Mixture-of-Mamba, a multi-modal SSM that leverages modality-aware sparsity for efficient multi-modal pretraining! At the core of Mixture-of-Mamba: 🔹Modality-aware sparsity to optimize efficiency 🔹Mixture-of-SSMs to enable cross-modal interactions 🔹Scales

Update: check out github.com/kfdong/STP for our code, data, and model!

We are excited to announce that Voyage AI is officially joining MongoDB ! Joining MongoDB enables us to bring our cutting-edge AI retrieval technology to a broader audience and seamlessly integrate it into mission-critical applications. Learn more: blog.voyageai.com/2025/02/24/joi…

🚀 Excited to share the most inspiring work I’ve been part of this year: "Learning to Reason without External Rewards" TL;DR: We show that LLMs can learn complex reasoning without access to ground-truth answers, simply by optimizing their own internal sense of confidence. 1/n

Thank you, Victoria X Lin , for the write-up.

Even the smartest LLMs can fail at basic multiturn communication Ask for grocery help → without asking where you live 🤦‍♀️ Ask to write articles → assumes your preferences 🤷🏻‍♀️ ⭐️CollabLLM (top 1%; oral ICML Conference) transforms LLMs from passive responders into active collaborators.