🌌🛰️Wanna know which features are universal vs unique in your models and how to find them? Excited to share our preprint: "Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment"! arxiv.org/abs/2502.03714 (1/9)

Made with Sora Input: KITTI image Prompt 1: “Make this into a semantic segmentation map” Prompt 2: “Make this into a depth map”

The sdxl-VAE models a substantial amount of noise. Things we can't even see. It meticulously encodes the noise, uses precious bottleneck capacity to store it, then faithfully reconstructs it in the decoder. I grabbed what I thought was a simple black vector circle on a white

1/n Introducing ReDi (Representation Diffusion): a new generative approach that leverages a diffusion model to jointly capture – Low-level image details (via VAE latents) – High-level semantic features (via DINOv2)🧵

EQ-VAE is accepted at #ICML2025 😁. Grateful to my co-authors for their guidance and collaboration! Ioannis Kakogeorgiou, Spyros Gidaris, Nikos Komodakis.

8. Multi-Token Prediction Needs Registers 🔑 Keywords: multi-token prediction, MuToR, language model, fine-tuning, generative tasks 💡 Category: Natural Language Processing 🌟 Research Objective: The paper introduces MuToR, a novel approach aimed at enhancing multi-token

1/n Multi-token prediction boosts LLMs (DeepSeek-V3), tackling key limitations of the next-token setup: • Short-term focus • Struggles with long-range decisions • Weaker supervision Prior methods add complexity (extra layers) 🔑 Our fix? Register tokens—elegant and powerful

Better LLM training? Gregor Bachmann & Vaishnavh Nagarajan showed next-token prediction causes shortcut learning. A fix? Multi-token prediction training (thanks Fabian Gloeckle) We use register tokens: minimal architecture changes & scalable prediction horizons x.com/NasosGer/statu…

🚀UniWorld: a unified model that skips VAEs and uses semantic features from SigLIP! Using just 1% of BAGEL’s data, it outperforms on image editing and excels in understanding & generation. 🌟Now data, model, training & evaluation script are open-source! github.com/PKU-YuanGroup/…

Achievement unlocked: having Alyosha at our FUNGI poster, the one person I had in mind when working on this paper on cheap and better representations for k-nn classification and not only #cvprinparis #cvpr2025

Just back from CVPR@Paris 🇫🇷, what a fantastic event! Great talks, great posters, and great to connect with the French & European vision community. Kudos to the organizers, hoping that it returns next year! 🤞 #CVPR2025 #CVPR2025

1/n 🚀New paper out - accepted at #ICCV2025! Introducing DIP: unsupervised post-training that enhances dense features in pretrained ViTs for dense in-context scene understanding Below: Low-shot in-context semantic segmentation examples. DIP features outperform DINOv2!

New paper out - accepted at #ICCV2025 We introduce MoSiC, a self-supervised learning framework that learns temporally consistent representations from video using motion cues. Key idea: leverage long-range point tracks to enforce dense feature coherence across time.🧵

Nice trick for fine-tuning with multi-token prediction without architecture changes: interleave learnable register tokens into the input sequence & discard them at inference. It works for supervised fine-tuning, PEFT, pretraining, on both language and vision domains 👇

Interesting alternative to multi-token prediction, though the figure is a bit unintuitive. Instead of attaching a head for each +d'th prediction, pass a dummy input token for each extra prediction through the model. This is A LOT more expensive, e.g. doing 2-step prediction