New CLIP model achieving 80.1% zero-shot ImageNet top1 from members of RAIVN lab, LAION and StabilityAI. This uses two prior methods from RAIVN: 1. Model soups, @mitchnw et al (arxiv.org/abs/2203.05482) 2. CuPL prompts (for 80.3%), Sarah Pratt et al (arxiv.org/abs/2209.03320)

Sharing our project on 1) accelerating and 2) stabilizing training for large language-vision models 1) Towards accelerating training, we introduce SwitchBack, a linear layer for int8 quantized training which matches bfloat16 within 0.1 for CLIP ViT-Huge arxiv.org/abs/2304.13013

Introducing DataComp, a new benchmark for multimodal datasets! We release 12.8B image-text pairs, 300+ experiments and a 1.4B subset that outcompetes compute-matched CLIP runs from OpenAI & LAION 📜 arxiv.org/abs/2304.14108 🖥️ github.com/mlfoundations/… 🌐 datacomp.ai

Excited to announce DataComp! Let us create stronger models using better-filtered data.

1/9 I am excited to announce that our workshop "Towards the Next Generation of Computer Vision Datasets" will be happening at ICCV 2023 in Paris. We will feature DataComp submissions, other data-centric papers, and invited talks by experts. datacomp.ai/workshop

Introducing💃AdANNS: A Framework for Adaptive Semantic Search🕺 TL;DR: Up to 90× faster nearest neighbor retrieval and 2× lower memory cost for web-scale search. Applies to vector search at scale & improves all "retrieval" augmented models! arxiv.org/abs/2305.19435 [1/8]

Check out our new work on improving large-scale nearest-neighbor search!

Checkout CREPE led by Zixian Ma, a new large-scale benchmark for vision-language model!! Drop by the poster at CVPR 2023.

If you are at #CVPR2023, come check out prompting-in-vision.github.io on Monday, June 19 from 9am - 12pm in West room 223-224. Speakers include Sarah Pratt from RAIVN lab as well as Ziwei Liu Phillip Isola Hyojin Bahng Ludwig Schmidt and Denny Zhou!

Check out this cool work led by Jiafei Duan on collecting robot data without a robot! 🤖🦾

Exciting work with open source code to facilitate research on medium sized language models! 🎉

Sharing some highlights from our work on small-scale proxies for large-scale Transformer training instabilities: arxiv.org/abs/2309.14322 With fantastic collaborators Peter J. Liu, Lechao Xiao, Katie Everett, many others (see final tweet!), Jaehoon Lee, Justin Gilmer, Simon Kornblith! (1/15)

E) The attention logit growth instability is still present when replacing softmax with pointwise alternatives. Side note: If you're interested in learning more about replacing softmax with a pointwise alternative like relu^2/√seqlen, checkout arxiv.org/abs/2309.08586! (12/15)

Check out🪆MatFormer🪆co-led by Aditya Kusupati: it’s a simple yet powerful general-purpose architecture with flexibility and elasticity built within. It works across modalities and enables super cool things at web-scale tasks🔥🔥

CLIP models have become a lot better since 2021

🎉 Very Excited to present our recent work on “Selective🔍 Visual Representations for Embodied-AI🤖” next week at ICLR in Vienna🇦🇹!! 📣📣Important update! Our code and pretrained models are now available through our project website 🌐: embodied-codebook.github.io🚀 👋Come to my

Very cool to see our RAIVN lab alum Rowan in the latest GPT-4o demo!!🚀🚀

Congrats [Dr. [Dr. Aditya] Kusupati]!!🪆🪆