Improving CLIP Training with Language Rewrites introduce Language augmented CLIP (LaCLIP), a simple yet highly effective approach to enhance CLIP training through language rewrites. Leveraging the in-context learning capability of large language models, we rewrite the text

New paper! We show how to leverage pre-trained LLMs (ChatGPT, Bard, LLaMa) to rewrite captions, and significantly improve over CLIP embeddings: arxiv.org/abs/2305.20088 Joint work with Yonglong Tian Phillip Isola (MIT), Dina Katabi (MIT) Lijie Fan (MIT)

StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners paper page: huggingface.co/papers/2306.00… We investigate the potential of learning visual representations using synthetic images generated by text-to-image models. This is a natural

Should you train your vision system on real images or synthetic? In the era of stable diffusion, the answer seems to be: synthetic! One stable diffusion sample can be worth more than one real image. paper link: arxiv.org/abs/2306.00984

🚀 Is the future of vision models Synthetic? Introducing SynCLR: our new pipeline leveraging LLMs & Text-to-image models to train vision models with only synthetic data! 🔥 Outperforming SOTAs like DinoV2 & CLIP on real images! SynCLR excels in fine-grained classification &

HNY! Excited to share SynCLR, that rivals CLIP and Dino v2 but uses pure synthetic data. The interesting part - it can outperform models (e.g. CLIP) directly trained on LAION-2B, which was the dataset used to train SD 1.5 that we used to generate images. arxiv.org/abs/2312.17742

Do we still need codebook/quantization for scalable autoregressive visual generation? No! Thrilled to share our latest work on scaling w/ continuous tokens. We observe power-law scaling behavior on val loss, and obtain SOTA coco FID and GenEval score. arxiv.org/abs/2410.13863

Check out our latest VQ-free text-to-image model, Fluid! At last, an autoregressive model can generate the face of the Mona Lisa, thanks to continuous token modeling 🤣.