Excited to give a talk on our work (w/ Javier Sagastuy mohammad hossein Rosa Cao Daniel Yamins) on inter-animal transforms at the CogCompNeuro Battle of the Metrics (5:15-7 pm EST)! We develop a principled approach to measuring similarity between DNNs and the brain. #CCN2024

Excited to present at #CCN2024! Join me and Daniel Yamins today at 11:15-1:15 (C54) for our poster: "Towards Task-Appropriate Readout Mechanisms For Physical Scene Understanding". We propose a novel strategy for designing task-appropriate readout models using idealized representations

New paper on self-supervised optical flow and occlusion estimation from video foundation models. Stefan Stojanov Jiajun Wu Seungwoo (Simon) Kim Rahul Venkatesh tinyurl.com/dpa3auzd @

Extracting structure that’s implicitly learned by video foundation models _without_ relying on labeled data is a fundamental challenge. What’s a better place to start than extracting motion? Temporal correspondence is a key building block of perception. Check out our paper!

Excited to share our recent work on self-supervised discovery of motion concepts with counterfactual world modeling. It has been a privilege to work on this project with amazing collaborators Stefan Stojanov Seungwoo (Simon) Kim David Wendt, Kevin Feigelis, Jiajun Wu and Daniel Yamins

Video prediction foundation models implicitly learn how objects move in videos. Can we learn how to extract these representations to accurately track objects in videos _without_ any supervision? Yes! 🧵 Work done with: Rahul Venkatesh, Seungwoo (Simon) Kim, Jiajun Wu and Daniel Yamins

New paper on 3D scene understanding for static images with a novel large-scale video prediction model. neuroailab.github.io/projects/lras_… Strong results in self-supervised depth extraction, novel view synthesis (aka camera control), and complex object manipulations.

🚀 Excited to share our new paper! We introduce the first autoregressive model that natively handles: 🎥 Novel view synthesis 🎨 Interactive 3D object editing 📏 Depth extraction ➕ and more! No fine-tuning needed—just prompting. Outperforming even diffusion-based methods!