🔥Telegram İfşa

Sharing my recent project, agent-to-sim: From monocular videos taken over a long time horizon (e.g., 1 month), we learn an interactive behavior model of an agent (e.g., a 🐱) grounded in 3D. gengshan-y.github.io/agent2sim-www/

Why is self-supervision in vision still not working? 🤔 When pretraining a transformer on TEXT-only data by predicting the next tokens, we see clear improvement trends as we scale the model, data, and computing. But after trying to pretrain a transformer on IMAGES-only data

We've released our paper "Generating 3D-Consistent Videos from Unposed Internet Photos"! Video models like Luma generate pretty videos, but sometimes struggle with 3D consistency. We can do better by scaling them with 3D-aware objectives. 1/N page: genechou.com/kfcw

We will present CoFie at #NeurIPS2024 tomorrow - a compact geometry-aware surface representation. CoFie disentangles the transformation of local patches and explicitly models it in SE(3), aligning local patches and reducing their complexity. Location: West Ballroom A-D #6900

#NeurIPS2024 In UniSDF, by unifying different neural representations, we achieved best overall performance across various scene types, ranging from object-level to unbounded scenes, with and without reflections. fangjinhuawang.github.io/UniSDF/

Ever wondered how roses grow and wither in your backyard?🌹 Our latest work on generating 4D temporal object intrinsics lets you explore a rose's entire lifecycle—from birth to death—under any environment light, from any viewpoint, at any moment. Project page:

💥 Think more real data is needed for scene reconstruction? Think again! Meet MegaSynth: scaling up feed-forward 3D scene reconstruction with synthesized scenes. In 3 days, it generates 700K scenes for training—70x larger than real data! ✨ The secret? Reconstruction is mostly

I was able to upload my own image to Veo 2! Here is the result when asking it to pan around Lofi Girl's room - extremely impressive result

Check out our new progress on large-scale visual localization with Scene Coordinate Regression (SCR)! In R-SCoRe, we close the gap between SCR and feature matching methods on challenging benchmarks with strong illumination changes. Paper: arxiv.org/pdf/2501.01421

Thanks Marco Pavone for sharing my internship work! It was a fantastic experience collaborating with you and your team at NVIDIA. DreamDrive is our preliminary exploration of driving everywhere leveraging the Internet street view images. Stay tuned for more updates!

Excited to share STORM! Unlike existing LRMs, STORM tackles dynamic scenes—reconstructing dynamic 3D scenes, estimating object velocities, and capturing different motion groups from a short video clip. Using a feedforward model, it slashes per-scene optimization time from 1000+s

📢 [New Paper] DMesh++: An Efficient Differentiable Mesh for Complex Shapes ✍️ Authors: Sanghyun Son, @gadelha_m, Yang, Matthew Fisher, Zexiang Xu, Yiling Qiao, Ming Lin, Yi Zhou 🔗 Arxiv: arxiv.org/abs/2412.16776 🔗 Page: sonsang.github.io/dmesh2-project/ More details👇

I'm very happy to announce that our paper "Don't Splat your Gaussians: Volumetric Primitives for Rendering Scattering and Emissive Media" (tinyurl.com/GaussVol) was finally accepted to ACM Transactions on Graphics last month! We will present it at SIGGRAPH 2025 🧵🧵🧵 (1/11)

Our project OmniRe has been accepted to ICLR 2025! 🎉 Huge thanks to all my fantastic collaborators for making this happen! 🙌 #ICLR2025 Project page: ziyc.github.io/omnire DriveStudio🚗: github.com/ziyc/drivestud…

Working on Depth Estimation? Here is a free lunch. We tune a Depth Anything ViT-B model on MegaSynth, and the performance improves a lot -- depth estimation is also very non-semantic! #CVPR2025 Accepted

Too many artifacts for GS reconstruction? Please checkout GenFusion: Closing the Loop between Reconstruction and Generation via Videos 🌐 Project page: genfusion.sibowu.com 💻 Code: github.com/Inception3D/Ge… #3D #DiffusionModels #ViewSynthesis #GenFusion #CVPR2025

Meet 𝐀𝐌𝐎 — our universal whole‑body controller that unleashes the 𝐟𝐮𝐥𝐥  kinematic workspace of humanoid robots to the physical world. AMO is a single policy trained with RL + Hybrid Mocap & Trajectory‑Opt. Accepted to #RSS2025. Try our open models & more 👉

3DGEER: Exact and Efficient Volumetric Rendering with 3D Gaussians Contributions: (i) We present the first complete, first-principle derived, closed-form solution for exact volumetric Gaussian rendering. (ii) We propose an exact and efficient ray-particle association method

Open Sourcing Forge: 3D Gaussian splat rendering for web developers! 3DGS has become a dominant paradigm for differentiable rendering, combining high visual quality and real-time rendering. However, support for splatting on the web still lags behind its adoption in AI.

🚀 Introducing GenLit – Reformulating Single-Image Relighting as Video Generation! We leverage video diffusion models to perform realistic near-field relighting from just a single image—No explicit 3D reconstruction or ray tracing required! No intermediate graphics buffers,