🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

New paper! We cast reward fine-tuning as stochastic control. 1. We prove that a specific noise schedule *must* be used for fine-tuning. 2. We propose a novel algorithm that is significantly better than the adjoint method*. (*this is an insane claim) arxiv.org/abs/2409.08861

As a speech/audio researcher, I think it’s a big breakthrough in advancing *human-level audio generation*. Why? Because this is NOT just a video-to-audio model that generates what it sees in the physical world... but a model that learns to *DESIGN* sounds like a human🤯 Here

Movie Gen claims to be the state-of-the-art in text-to-video generation, outperforming Sora, Kling, Gen3, and more. But how can you trust the results? Today, we're releasing 1003 videos and their prompts - no cherry-picking allowed. Our goal? To set a new standard for evaluating

MovieGen is now on arXiv, with some interesting new tidbits! I’m particularly excited about this scaling analysis, where we find that the optimal FLOPs/params for MovieGen lie on the Llama3 scaling law, suggesting that LLM scaling laws might even work for media generation models!

🚨 Internship in Meta GenAI NYC 🚨 I have an open PhD internship position for 2025! Interested in exploring visual generative models (or any other exciting ideas) inside the team that brought you Movie Gen and Emu Video? 📩 Send me DM with CV, website, and GScholar profile

Ever wondered why current SOTA gen models learn a mapping all the way from noise to the target distribution? Well, turns out that this is a constraint of diffusion models, but NOT flow matching models! We explore this in cross-flow.github.io ! really enjoyed this one 😄

✨New blog post✨: my attempt as a vision researcher at finally understanding RLHF -- a deep dive into PPO & DeepSeek's GRPO! No hot take, I promise. yugeten.github.io/posts/2025/01/…

Super excited to share some recent work that shows that pure, text-only LLMs, can see and hear without any training! Our approach, called "MILS", uses LLMs with off-the-shelf multimodal models, to caption images/videos/audio, improve image generation, style transfer, and more!

VideoJAM is our new framework for improved motion generation from AI at Meta We show that video generators struggle with motion because the training objective favors appearance over dynamics. VideoJAM directly adresses this **without any extra data or scaling** 👇🧵

Video JEPA learns intuitive physics.

This is a really great opportunity to work with Tengda and the team!

Thanks to the organizers for inviting me! Tune in at 3pm PDT here for my talk about how transformers have changed the game for video generation