(1/N) CFG requires high guidance (>5) to "work", but comes with several issues 🤦‍♂️: reduced diversity, saturation, poor invertibility. Is this inevitable? 🤔 Presenting CFG++,🚀 a simple fix enabling small guidance: better sample quality + invertibility, smooth trajectory 🤟

Random note of: our paper "Harmonic Self-Conditioned Flow Matching for Multi-Ligand Docking and Binding Site Design" was accepted at ICML Conference Much party, such celebration 🙃 arxiv.org/abs/2310.05764 Well, let me know if you are at ICML and think chatting could be interesting :)

I am happy to announce that NVIDIA's own ViT for image generation (DiffiT) has been accepted to ECCV 2024 🎉 🥂 Paper: arxiv.org/pdf/2312.02139 Code (soon): github.com/NVlabs/DiffiT collaborators: Arash Vahdat Jan Kautz Jiaming Song Guilin Liu

Introducing Discrete-Continuous Latent Variable Diffusion Models (DisCo-Diff 🕺), which augment continuous diffusion models with learnable global discrete latents. DisCo-Diff greatly simplify learning diffusion models and strengthens their sampling trajectories (1/9)

🕺🪩 DisCo-Diff! 🪩🕺 Model your data with continuous latent variables, as in diffusion models? Or with discrete tokens? Do both! Augment your diffusion model with discrete latents and get the best of both! 🔥 w/ the amazing Yilun Xu, Gabriele Corso, Tommi Jaakkola, Arash Vahdat

Check out our latest work! 👇We leverage pretrained protein generative models (here Chroma Generate:Biomedicines ) as a prior for inverse problems in protein space (e.g. structure completion, distance contraints, cryo-EM model building). ❄️🏗️ Paper: arxiv.org/abs/2406.04239

I got a very loud lesson today on the difference between psi vs. bar.

DisCo-Diff 🕺🪩 was developed by Yilun Xu, Gabriele Corso, #JClinic PI Tommi Jaakkola, Arash Vahdat, & Karsten Kreis to enhance the diffusion models by making the learning process easier + more efficient to generate high quality, complex data Paper: arxiv.org/pdf/2407.03300

🎉 Today I taught my first class on "How to Create a Protein"! 🧬👩‍🏫 High School students learned about proteins and protein design, and even got to create their own brand-new proteins using "Frankeinstein" and Chroma! 🧪🔬✨ The proteins can then be 3D printed! 🖨️ (1/2)

🚀 40x Faster Model Training via Pruning and Distillation! Permissive Minitron-4B and Minitron-8B models! 🔗 Paper: arxiv.org/abs/2407.14679 🔗 GitHub: github.com/NVlabs/Minitron 🔗 Models on HF: bit.ly/4ffjnQj Key highlights of 4B/8B models: 📊 2.6B/6.2B active

🔥🔥🔥

My main points at the #ICML2024 AI for Science panel (general guidelines, exceptions exist): -- Good projects are structured to have intermediate milestones that are measurable, thus creating constructive feedback loops. (I knew this one would get to Max Welling 😀) --

i was going thru notes on my phone and found this quote and i have no idea where it's from lol it's honestly good advice though lol

Applications are now open for the NVIDIA graduate fellowship awards. This is a fantastic opportunity to get support for your cutting edge PhD research and get to know NVIDIA Research: blogs.nvidia.com/blog/applicati…

🌀 Hurricane help from #NVIDIAResearch: StormCast is a new #generativeAI model for emulating high-fidelity atmospheric dynamics. This model can enable reliable weather prediction at mesoscale which is critical for disaster planning and mitigation. ➡️ nvda.ws/3X9AsEx ⛈️

🔥 Very cool work by our colleagues at NVIDIA!

According to amanda silberling of TechCrunch, AI can't spell Strawberry. Here is Ideogram's output for the following prompt: A collection of strawberries against a pink background. There's a unique strawberry in the center that has been sliced open, revealing its juicy interior.

🔥🔥Our new #SIGGRAPHAsia2024 paper shows how compositional text-to-image diffusion models (BlobGEN) can be used for dragging objects around in an image.

⚡ Ray Tracing + 3D Gaussians = New Possibilities! Gaussian splatting is limited by rasterization—our #SIGGRAPHAsia2024 paper shows how to ray trace instead, enabling reflections, shadows, fisheye cameras, and more. The most important (and hardest!) part is making it fast. (1/N)

📢 Life update: I've joined AI at Meta (#FAIR #NYC) as #ResearchScientist this week 🍎! Extremely grateful for everyone who's supported me along the way 🙂 I'll keep working on flow/diffusion for structural problems, Schrodinger bridges, optimization, stochastic control, & more🏃🏻‍♂️