Sorry if somebody did this one before - but the field is growing so fast, there is no way I can keep track of it! While making this I constantly felt like somebody is scooping me - or was I already scooped? #ComputerVision #CVPR2021 #TypesofPaper

Types of Computer Vision Paper (source: Randall Munroe)

[3/3] Towards big vision While dense models are still the norm, sparse MoE layers can work well too! Large Vision-MoEs (15B params) can be trained to high performance relatively efficiently, and can even prioritize amongst patches (see duck). arxiv.org/abs/2106.05974 ...

Check out what we’ve been working on for the last months: We decouple the model size and the compute cost in a Vision Transformer backbone by using Sparse MoE layers. These have been popularised in NLP, and they are fantastic for Vision too! arxiv.org/abs/2106.05974

Unsure which arch to use for your deep ensemble? Why settle for one? Neural Ensemble Search constructs ensembles with varying network archs Paper: arxiv.org/abs/2006.08573 Code: github.com/automl/nes Work by Sheheryar Zaidi Arber Zela Thomas Elsken Chris Holmes Frank Hutter Yee Whye Teh

How to do research with my mentors effectively? I get this question frequently in my open office hours. I am still learning as well but I hope sharing my ✌💰 may be helpful to some. Key idea ➡️ **Help them help you!** How? Check out the thread 🧵

MICo: Improved representations via sampling-based state similarity for MDPs Our #NeurIPS2021 paper introduces a new loss that improves your RL agents! 📜Paper: arxiv.org/abs/2106.08229 💻Blog: psc-g.github.io/posts/research… 🐍Code: github.com/google-researc… 1/🧵

New blog post! Some thoughts about diffusion distillation. Actually, quite a lot of thoughts 🤭 Please share your thoughts as well! sander.ai/2024/02/28/par…

🎓Kullback-Leibler divergence between densities of an exponential family = reverse Bregman divergence wrt the cumulant function 🎉Kullback-Leibler divergence between non-normalized densities = reverse Bregman divergence wrt the partition function 👉 arxiv.org/abs/2312.12849

me and my supervisor discussing how our predictions are worse than random

Stein's unbiased risk estimate (SURE) is an almost magical formula that enables the computation of the mean squared error of a denoiser (used, for example, in denoising score matching) using only the noisy observation y, without requiring the clean data x. en.wikipedia.org/wiki/Stein%27s…