I’m excited to give a short talk on Video Motion Customization (video-motion-customization.github.io) and related works tomorrow with Hyeonho Jeong. Huge thanks to James Le for hosting us! Feel free to register: mailchi.mp/twelvelabs/mul…

One paper accepted at #Neurips2024. Direct Unlearning Optimization for Robust and Safe Text-to-Image Models: arxiv.org/abs/2407.21035 I'm really glad to achieve a good result from my first internship NAVER AI LAB. I'd like to express my gratitude to all my collaborators.

Why are there so many different methods for using diffusion models for inverse problems? 🤔 And how do these methods relate to each other? In this survey, we review more than 35 different methods and we attempt to unify them into common mathematical formulations.

ViBiDSampler Enhancing Video Interpolation Using Bidirectional Diffusion Sampler

TweedieMix Improving Multi-Concept Fusion for Diffusion-based Image/Video Generation

No distillation, no perceptual losses - no problem! Check out recent work on fast flows with ReFlow by Beomsu Kim - Key choices: high pass filter loss, loss weight, dropout, time/step schedules, real data coupling - Insights into struggles with OT methods

🚨 Introducing FreeMCG🆓 1️⃣ Unified🤝 XAI for feature attribution (FA) & counterfactuals (CF) 2️⃣ On-manifold🍩 changes that swiftly avoid adversarial attacks, using diffusion models 3️⃣ All this, derivative-free🤯 i.e. black-box classifier is all you need A 🧵

#Track4Gen: Video diffusion training (finetuning) with dense point tracking loss on diffusion features! Paper: arxiv.org/abs/2412.06016 Project: hyeonho99.github.io/track4gen Work done during my internship at Adobe with amazing collaborators: Duygu Ceylan Paul Huang &Niloy Mitra

Does conditioning on metadata (i.e. patient demographic, MR imaging params, pathology) help MRI recon? 🤔 Yes! 👍 🚨Announcing ContextMRI⚕️, a systematic study clarifying the effect and the impact of each component of the metadata A 🧵

If you want to diffuse stuff, its frequency behaviour is important🌊 (sander.ai/2024/09/02/spe…). For latents, you can shape the spectrum! Like EQ-VAE, they find: equivariance ⇒ better latents. Loving all the recent work on tweaking latents, might be time for another blog post✍️

Aligning Text to Image in Diffusion Models is Easier Than You Think "SoftREPA provides much improved text to image alignment by introducing a negligible size of learnable soft tokens."

3D consistent videos are hard to generate 🙁 What if we could steer them to be consistent during generation? Introducing SteerX🛞, a plug-and-play sampling method that works with *any* video diffusion to make videos physically plausible🤩 w/ Byeongjun Park Hyojun Go Hyelin Nam

Prompt Theory (Made with Veo 3) What if AI-generated characters refused to believe they were AI-generated?

Chain-of-Zoom Extreme Super-Resolution via Scale Autoregression and Preference Alignment