Generating nice meshes in AI pipelines is hard. Our #SIGGRAPHAsia2024 paper proposes a new representation which guarantees manifold connectivity, and even supports polygonal meshes -- a big step for downstream editing and simulation. (1/N) SpaceMesh: research.nvidia.com/labs/toronto-a…

📢🚗✨ Excited to announce InfiniCube, our scalable generative model for dynamic 3D driving scene generation with high fidelity and controllability! InfiniCube generates very large-scale (300m×400m ~ 100,000m^2), dynamic 3D driving scenes given HD maps, 3D bounding boxes, and

🚀 Introducing DiffusionRenderer, a neural rendering engine powered by video diffusion models. 🎥 Estimates high-quality geometry and materials from videos, synthesizes photorealistic light transport, enables relighting and material editing with realistic shadows and reflections

Leveraging diffusion prior to 3D reconstruction at both training and inference enables larger viewpoint changes and less artifacts!! 🚀📷

🚀Excited to introduce GEN3C #CVPR2025, a generative video model with an explicit 3D cache for precise camera control. 🎥It applies to multiple use cases, including single-view and sparse-view NVS🖼️ and challenging settings like monocular dynamic NVS and driving simulation🚗.

🚀Excited to release PartField—a feedforward model that learns part-based feature fields for 3D shapes! It enables lightning-fast⚡️, robust, open-world hierarchical 3D part seg and unlocks cross-shape applications like co-seg and correspondence! 🔗shorturl.at/HnUmc 1/n

Learning to generate a 3d feature field enables hierarchical decomposition for open-world shapes with robustness and efficiency! All achieved with one simple loss function. The correspondence across shapes surprisingly emerges. Code and models are released for you to try

This year, we have 3 papers in CVPR, discussing the connection between 3D and video models: GEN3C [Highlight] 3D grounding for video model DiffusionRenderer [Oral] Taming video models for rendering and inverse rendering Diffix3D+ [Oral] Enhancing Nerf/3DGS w/ diffusion models

🚀 Difix3D+ is now open-sourced! Check out the code and try the demo: github.com/nv-tlabs/Difix… We're presenting at #CVPR2025 this Sunday, June 15 — come say hi! 🗣️ Oral: 1:00–1:15 PM CDT, Karl Dean Grand Ballroom 🖼️ Poster: 4:00–6:00 PM CDT, ExHall D (Poster #57)

🚀 We’re really excited to share the code and model for GEN3C now! This release brings @NVIDIA’s cosmos to GEN3C with better visual quality, generalizability, and longer video sequence! 🥳 We also released the GUI to allow users specify camera trajectory in the generated video.

Code and model for Difix3D+ is also released!

More tests last night with NVIDIA AI Developer's GEN3C. This was all generated from a single photo using local compute. I love how it accurately simulated the light fade around the building corner. Next up, dynamic scenes. #3D #GenAI #Computervision

On a scale of 1-10, 10 being best, how well did NVIDIA AI Developer's GEN3C handle that large mirror reflection? I think it did a great job! Only one photo was used for input. #3D #Computervision #GENAI

Thanks for trying this! We do have an open-source model that people can try to explore the "3D" worlds with video generative models. Code & Model: github.com/nv-tlabs/GEN3C runnable on one workstation GPU!!

[1/N] 🎥 We've made available a powerful spatial AI tool named ViPE: Video Pose Engine, to recover camera motion, intrinsics, and dense metric depth from casual videos! Running at 3–5 FPS, ViPE handles cinematic shots, dashcams, and even 360° panoramas. 🔗 research.nvidia.com/labs/toronto-a…

[6/N] ViPE is already in action. For example: ➕Powering controllable video generation in #Gen3C and #Cosmos. ➕Providing geometry loss for feed-forward reconstruction in #BTimer. We’re excited to see how the community will build on ViPE for spatial AI and robotics!

Understanding camera pose for in-the-wild dynamic videos is an essential building block for spatial AI. Today, we release a very well-engineered and designed system for it led by Jiahui Huang Code has been released (just one line cmd to try) with a large scale dataset!

ViPE powers Gen3C for controllable video generation!