📢 Check out 𝐒𝐚𝐩𝐢𝐞𝐧𝐬 (#ECCV2024, Oral)! Largest human-centric foundational models that natively supports 1K high-res inputs, achieving SOTA on 2D kpts, seg, depth and normal! Code and models (each task + pretraining) are ready to use TODAY! github.com/facebookresear… (1/6)

We present InterTrack: tracking human-object interaction without any object template. We reconstruct shapes from scratch and track them from a single RGB video. Project page: virtualhumans.mpi-inf.mpg.de/InterTrack/ Arxiv: arxiv.org/abs/2408.13953 With Xianghui Xie Jan Eric Lenssen

Many works model 3D virtual humans, but how can we create realistic dense crowds? At SIGGRAPH Asia 2024, we will show that solving human-human interactions is key to modeling realistic crowds. project: mslab.es/projects/Dense… 🧵1/4

Excited to share our DepthCrafter, a super consistent video depth model for long open-world videos! Project webpage: depthcrafter.github.io

Are 3D human pose estimators robust to clothing, location, weather etc.? Nobody knows, as current benchmarks can't answer this. We therefore introduce STAGE, a GenAI toolkit to create custom benchmarks, and use it to reveal weaknesses of SOTA models in the open world.

🚀 Announcing GTA-Human II for expressive human pose & shape estimation! Compared to its predecessor, this latest game-playing dataset features: ✅ Multi-person scenes ✅ SMPL-X annotations (upgraded from SMPL) ✅ Point cloud data 🔗 Homepage: caizhongang.com/projects/GTA-H…

#ECCV24 We introduce Controllable Human-Object Interaction Synthesis (CHOIS) to generate interactions given text in a 3D scene. w alex, Roozbeh Mottaghi Jiajun Wu Xavier Puig Karen. code: github.com/lijiaman/chois… project page: lijiaman.github.io/projects/chois/

Excited to share MonST3R! -- a simple way to estimate geometry from unposed video of dynamic scene We achieve competitive results on several downstreams (video depth, camera pose) and believe this is a promising step toward feed-forward 4D reconstruction monst3r-project.github.io

🔥 Introducing MVLift: Generate realistic 3D motion without any 3D training data - just using 2D poses from monocular videos! Applicable to human motion, human-object interaction & animal motion. Joint work w/ Jiajun Wu & Karen 💡 How? We reformulate 3D motion estimation as

🚀 Introducing DiffusionRenderer, a neural rendering engine powered by video diffusion models. 🎥 Estimates high-quality geometry and materials from videos, synthesizes photorealistic light transport, enables relighting and material editing with realistic shadows and reflections

🐅 Want to rig your favorite meme character? Try “RigAnything: Template-Free Autoregressive Rigging for Diverse 3D Assets”! ✨RigAnything is a transformer-based model that sequentially generates skeletons without predefined templates. It creates high-quality skeletons for

Test-Time Training (TTT) is now on Video! And not just a 5-second video. We can generate a full 1-min video! TTT module is an RNN module that provides an explicit and efficient memory mechanism. It models the hidden state of an RNN with a machine learning model, which is updated

LLMs treat tokens like currency. What if the social internet did too? Fewer words, higher value. Would we become better communicators—or just quieter? 💭

I'm moving cities, so I made a furniture sale spreadsheet. It started as a way to stay organized, but ended up being super handy for sharing with buyers. A stranger on Marketplace said they dug it, which ended up being the highlight of my week 😂

Ever have a thought, check Reddit, and: (A) the crowd agrees—so you feel validated and move on (B) you're in the minority—and now you're either annoyed... or rethinking everything? Funny how total strangers on the internet can sway how you feel (or what you think) 🧠

Talking to friends as user-study “participants” is so fun -- you get to see everyone’s hidden talk show guest energy come out🎙️😂

TIL the whole "like this." vs "like this". thing is actually a US vs UK difference, not just AI writing tools being weird 😅 #GrammarWars