Really impressive results. I think data driven approaches will be able to do fully inverse/forward rendering soon, including strong specular effects, hard shadows and transparencies.

Introducing Discrete-Continuous Latent Variable Diffusion Models (DisCo-Diff 🕺), which augment continuous diffusion models with learnable global discrete latents. DisCo-Diff greatly simplify learning diffusion models and strengthens their sampling trajectories (1/9)

Diffusion Forcing Update: code & ckpt for 3D-UNet + Temporal Attention version is released thanks to my amazing undergrad mentee Kiwhan Song! See project github for more info. I also added suggested future directions to the our website boyuan.space/diffusion-forc…. Check them out!

Got a chance to play pingpong in VR with this virtual agent on May, it’s so cool! Imagine more sophisticated interactions with virtual agent in the future.

So cool

Want to turn an IKEA lamp into an intelligent robot capable of learning and adapting autonomously by simply adding cameras and motors? Excited to share our multi-disciplinary project that blends robotics, computer vision, and next-gen fabrication to make it possible!

PhysDreamer has been accepted by ECCV with *Oral* presentation🌹🎉. Check out Tianyuan Tianyuan Zhang 's wonderful introduction👇

Excited to announce that I will be joining the University of Cambridge Engineering Dept as an assistant professor in spring 2025! I will be looking for students for the next year. Check out Elliott / Shangzhe Wu's thread for details on how to apply, and get in touch!

Really love this line of works!

Very cool results!

Super excited to see Adobe Firefly’s Fast Mode out in the wild! 🎨 Thrilled to have contributed to the research that helped make this a reality. Can’t wait to see how creators unleash their imagination with it! 🚀 #AdobeFirefly

Join us next Friday on October 11th at 10 AM PT for our next LIVE PyTorch Expert Exchange Webinar on Efficient Streaming Language Models with Attention Sinks w/ Guangxuan Xiao, MIT EECS ▶️🎙️ Tune in at: hubs.la/Q02SfV5v0 #LLMs

I'm recruiting PhD students this year with interest in machine learning, embodied AI, or AI for science! If you are interested in constructing fundamental tools to improve Generative AI and exploring how these tools can be used for intelligent embodied agents and science,

Seems that only a few “retrieval” head needs KV cache.

Accurate and controllable scene generation has been difficult with natural language alone. You instead need a language for scenes. Introducing the Scene Language — a visual representation for high-quality 3D/4D generation by integrating programs, words, and embeddings — 🧵(1/6)

Novel view synthesis has long been a core challenge in 3D vision. But how much 3D inductive bias is truly needed? —Surprisingly, very little! Introducing "LVSM: A Large View Synthesis Model with Minimal 3D Inductive Bias"—a fully transformer-based approach that enables scalable,

Check our learning-based deterministic model for novel view synthesis without NeRF and 3DGS, yet producing consistent rendering(still need camera poses). We try to use few 3D inductive biases and make it simple!

Current vision systems use fixed-length representations for all images. In contrast, human intelligence or LLMs (eg: OpenAI o1) adjust compute budgets based on the input. Since different images demand diff. processing & memory, how can we enable vision systems to be adaptive ? 🧵

Humans can learn to reason in an "unfamiliar" world, like new games. How far are LLMs from this? Check out our recent work @NeurIPS2024 D&B Track: "LogiCity: Advancing Neuro-Symbolic AI with Abstract Urban Simulation". Page: jaraxxus-me.github.io/LogiCity/