🚨 I’ll be at #NeurIPS2024! 🚨On the industry job market this year and eager to connect in person! 🔍 My research explores multimodal learning, with a focus on object-level understanding and video understanding. 📜 3 papers at NeurIPS 2024: Workshop on Video-Language Models 📅

📢Come to join our 1st Workshop on Video-Langauge Models at #NeurIPS 2024. We have seen a great progress on image-language models, now it is time for Videos! Our invited speakers will talk more about how we further move forward! …and-language-workshop-2024.webflow.io Special invited talks

🔥Poster: Fri 13 Dec 4:30 pm - 7:30 pm PST (West) It is the first time for me try to sell a new concept that I believe but not in trend. I truely trust the language between llm/lmms are embeddings, and interfacing with embeddings is essential in future! Welcome everyone to come😀

Check out our new ICLR 2025 paper, LLaRA, which transforms a pretrained vision-language model into a robot vision-language-action policy! Joint work with Xiang Li, Michael Ryoo, et al from Stony Brook U, and Mu Cai. github.com/LostXine/LLaRA

Congratulations again Mu Cai!! So well deserved. I will miss having you in the lab.

Public service announcement: Multimodal LLMs are really bad at understanding images with *precision*. x.com/lukeprog/statu… A thread🧵: 1/13.

🚀 Excited to announce our 4th Workshop on Computer Vision in the Wild (CVinW) at #CVPR2025 2025! 🔗 computer-vision-in-the-wild.github.io/cvpr-2025/ ⭐We have invinted a great lineup of speakers: Prof. Kaiming He, Prof. Boqing Gong, Prof. Cordelia Schmid, Prof. Ranjay Krishna, Prof. Saining Xie, Prof.

Congratulations Dr. Mu Cai Mu Cai! Mu is my 8th PhD student and first to start in my group at UW–Madison after my move a few years ago. He made a number of important contributions in multimodal models during his PhD, and recently joined Google DeepMind. I will miss you a lot Mu!

Thank you AK for sharing our work!

Training text-to-image models? Want your models to represent cultures across the globe but don't know how to systematically evaluate them? Introducing ⚕️CuRe⚕️ a new benchmark and scoring suite for cultural representativeness through the lens of information gain (1/10)

LLaVA-Prumerge, the first work of Visual Token Reduction for MLLM, finally got accepted after being cited 146 times since last year. Congrats to the team! Yuzhang Shang Yong Jae Lee See how to do MLLM inference much cheaper while holding performance. llava-prumerge.github.io

#ICCV2025 Introducing X-Fusion: Introducing New Modality to Frozen Large Language Models It is a novel framework that adapts pretrained LLMs (e.g., LLaMA) to new modalities (e.g., vision) while retaining their language capabilities and world knowledge! （1/n） Project Page:

My students called the new CDIS building “state-of-the-art”. I thought they were exaggerating. Today I moved in and saw it for myself. Wow. Photos cannot capture the beauty of the design.

Here is the final decision for one of our NeurIPS D&B ACs-accepted-but-PCs-rejected papers, with the vague message mentioning some kind of ranking. Why was the ranking necessary? Venue capacity? If so, this sets a concerning precedent. NeurIPS Conference