A group of Flamingos is called “flamboyance” which could be an apt description for the family of vision-language models I’m thrilled to see out in the wild! I believe using large pre-trained models in creative ways will be key and hope our work is a step in the right direction.

Great to finally share our newest addition to the DeepMind large-scale model zoo! 🦩

A lot happenned in the last year ! I defended my PhD and joined @DeepMind where I worked with an incredible team on Flamingo🦩, a visual language model. Flamingos can fly, they can dance, and this one writes pretty well too !

Was a total pleasure to be part of the team for Flamingo. Lots of exciting capabilities that we are just beginning to explore.

Chatting with Flamingo about images is definitely the most organic experience I’ve had with an ML model. The ability to readily describe output from e.g. DALL-E 2 might be the closest we’ve come to two independently-trained large-scale models having a conversation 👀

10B extra parameters for adaptation and visual conditioning, new cross-modality data and a lot of love makes Chinchilla able to see !

Finally able to share what I have been working on this year! 🦩 Tldr: We took our best LM (Chinchilla), froze it and added new visual layers to it and trained 🦩 on full webpages with images instead of just image-text pairs. Check out the visual dialogue examples from the paper!

In case you missed it...Flamingo 🦩 a new SOTA visual language model. Read more below ⬇️ Paper: dpmd.ai/dm-flamingo-pa… Blog: dpmd.ai/dm-flamingo

Introducing Vid2Seq, a new visual language model for dense video captioning. To appear at #CVPR2023. Work done Google w/ Arsha Nagrani P.H. Seo Antoine Miech Jordi Pont-Tuset I. Laptev J. Sivic Cordelia Schmid. Page: antoyang.github.io/vid2seq.html Paper: arxiv.org/abs/2302.14115 🧵/5

🦩 Introducing OpenFlamingo! A framework for training and evaluating Large Multimodal Models (LMMs) capable of processing images and text. More details below (including a multimodal LLaMA model!)⬇️ Blog: laion.ai/blog/open-flam… Demo: 7164d2142d11.ngrok.app

Okay, so that AI letter signed by lots of AI researchers calling for a "Pause [on] Giant AI Experiments"? It's just dripping with #Aihype. Here's a quick rundown. >>

Demystifying CLIP Data Reveals CLIP’s data curation approach and makes it open to the community repo: github.com/facebookresear… abs: arxiv.org/abs/2309.16671

Thrilled to share #Lyria, the world's most sophisticated AI music generation system. From just a text prompt Lyria produces compelling music & vocals. Also: building new Music AI tools for artists to amplify creativity in partnership w/YT & music industry deepmind.google/discover/blog/…

Our latest release Mistral AI Mixtral 8x7B mixture of experts - performance of a GPT3.5 - inference cost of a 12B model - context length of 32K - speaks English, French, Italian, German and Spanish Blog post mistral.ai/news/mixtral-o…

Mixtral 8x7B is here, 11 weeks only after Mistral 7B. Outperforms Llama 2 70B and GPT 3.5 on most benchmarks, at the inference cost of a 12B dense model, with 32k tokens context size.

Thrilled to present to you Gemma! A family of lightweight, state-of-the art and open models by Google DeepMind. We provide both pre-trained and fine-tuned checkpoints for easy tuning, responsible development, and community-driven innovation! More info at ai.google.dev/gemma

We present a new SOTA on point tracking, via self-supervised training on real, unlabeled videos! BootsTAPIR achieves 67.4% AJ on TAP-Vid DAVIS with minimal architecture changes, tracks 10K points on a 50-frame video in 6 secs. Pytorch & JAX impl on Github. bootstap.github.io

We're excited to release TAPVid-3D: an evaluation benchmark of 4,000+ real world videos and 2.1 million metric 3D point trajectories, for the task of Tracking Any Point in 3D!

So pleased and proud to share with you what our team has been up to, on an ambitious journey to build a video foundation model for scientific domains ! ✨ 🚀 🎞️ 🧪 #ICCV2025 #AI4Science

Scaling 4D Representations – new preprint arxiv.org/abs/2412.15212 and models now available github.com/google-deepmin…