Just under a month until the testing stage of VISDA 2022. There’s still time to submit! Help push vision-based recycling sorting forward, get up to a $2000 prize, and get a chance to present at our workshop. More details in pinned tweet and at ai.bu.edu/visda-2022/.

📢📢Reminder that the virtual workshop for the VISDA-2022 challenge is 4PM ET tomorrow (Thursday)! We will be announcing winners and have a fantastic speaker lineup of 5 academic and industry leaders for AI in recycling and science. ai.bu.edu/visda-2022/

New Creator Katherine Sizov is bringing data to supply chains to drastically reduce food waste and climate change. See why she believes data can fuel the next wave of social and business progress. #LetsCreate

wow!

✏->📷 Our most recent work, MaskSketch, takes a sketch and turns it into a realistic photo, without any training. It even works on amateur drawings. The key is guiding a pretrained MaskGIT model with a structure preserving loss.

So we are finally launching Avaturn. 🚀Well, we actually already posted this a month ago and we managed to drop a database on backend in 5 minutes after the launch. True story, but finally it's 100% live and will not crash again. Support us by sharing!

COLA: How to adapt vision-language models to Compose Objects Localized with Attributes? abs: arxiv.org/abs/2305.03689 paper pages: huggingface.co/papers/2305.03…

Stable Diffusion and other text-to-image models sometimes blatantly copy from their training images. We introduce Ambient Diffusion, a framework to train/finetune diffusion models given only *corrupted* images as input. This reduces the memorization of the training set. A 🧵

Excited to share our new work on reducing copying in diffusion models. We proposed ways to mitigate copying behavior even in the presence of heavy training data duplication! Stay tuned for the TLDR thread!

Excited to share self-guidance, a new method for controllable image generation that guides sampling using only the attention and activations of a pretrained diffusion model: dave.ml/selfguidance Work led by Dave Epstein w/A Jabri, Ben Poole, Alyosha Efros More in thread🧵

Predicting masked tokens in stochastic locations improves masked image modeling paper page: huggingface.co/papers/2308.00… Self-supervised learning is a promising paradigm in deep learning that enables learning from unlabeled data by constructing pretext tasks that require learning

We propose a new type of diffusion model that diffuses continuous functions: Functional Diffusion. Biao Zhang (Biao Zhang ) and Peter Wonka. arXiv arxiv.org/abs/2311.15435 1zb.github.io/functional-dif…

Guidance on top of diffusion models can now be used to drag and manipulate images, create pose-conditioned images, and so much more! Check out Readout Guidance: readout-guidance.github.io Work w/ trevordarrell, Oliver Wang, Dan Goldman, Aleksander Holynski. More in thread 🧵.

Introducing💥Lasagna: a layered diffusion model for image relighting. Lasagna adds realistic lighting to input images, even to vector art! Joint work with Dina Bashkirova Arijit Ray Rupayan Mallick, Sarah Adel Bargal Ranjay Krishna Jianming Zhang arxiv.org/abs/2312.00833

Excited to be at NeurIPS! Presenting our poster on evaluating & adapting VLMs for multiple attribute-object relationships. Joint work with AI at Meta- Kate Saenko, Ranjay Krishna, Filip Radenovic, Abhimanyu Dubey, Bryan Plummer. cs-people.bu.edu/array/research…

Kidnappers returning me after I talk about my research for 1 hour

My group at FAIR (Meta) is looking for a postdoc in vision and language! Please apply here metacareers.com/jobs/141798883…

🐨Koala: Key frame-conditioned long video-LLM Koala is a new video-LLM that can answer questions about longer videos than previously possible. --with R. Tan, X. Sun, P. Hu, J. Wang, H. Deilamsalehy, B.A. Plummer and B. Russell Link to paper and demo below

Excited to introduce our new paper, Generative Omnimatte: Learning to Decompose Video into Layers, with the amazing team at Google DeepMind! Our method decomposes a video into complete layers, including objects and their associated effects (e.g., shadows, reflections).