What if I tell you that you can tune the embeddings of any pre-trained model to have Matryoshka properties, both in unsupervised and supervised settings? I finished reading the latest paper Matryoshka-Adaptor from Google. Here is a quick summary... 1. Why worry about embeddings?

Hello! For my biennial post here, a bit of a personal update: after 5 great years at Microsoft Research, this fall, I'll be joining the CSE faculty at University of Washington. Much love to the great folks at MSR I've had the pleasure of working with. 1/2

Hey Guys, I'm gonna present LLM in a flash in ACL 2024. Hit me up if you are in Bangkok. arxiv.org/abs/2312.11514 Updates from previous version: - Llama 2 results - Some results on Apple GPUs (Metal) - Speculative decoding - Memory Latency Tradeoff - Impact of longer generation

The application is now open for our #KempnerInstitute Research #Fellowship! Postdocs studying the foundations of #intelligence or applications of #AI are encouraged to apply. Learn more and apply by Oct. 1: bit.ly/4djPSf8 #LLMs #NeuroAI #ML Sham Kakade Bernardo Sabatini

We’re introducing Gemini Live, a more natural way to interact with Gemini. You can now have a free-flowing conversation, and even interrupt or change topics just like you might on a regular phone call. Available to Gemini Advanced subscribers. #MadeByGoogle

Meet Gemini Live: a new way to have more natural conversations with Gemini. 💬 💡 Brainstorm ideas ❓ Interrupt to ask questions ⏸️ Pause a chat and come back to it Now rolling out in English to Gemini Advanced subscribers on Android phones → dpmd.ai/46RToL9

The never-ending saga of NeurIPS Conference reviews continues with no clear resolution in sight. Rebuttal is nice, keep the score of borderline! Rebuttal works, need a new expt from 4 days ago! Rebuttal makes sense, I don't beleive it! So much effort wasted into these numbers! 🥱

A big day for Search and our users in the UK, India, Japan, Indonesia, Mexico and Brazil! We’re expanding AI Overviews to these 6 countries with local language support. 📷

Exceptional work from Pavlo Molchanov and team!!

Go use an even faster Flash!⚡

Gemini 1.5 Flash is now tied with frontier models. Here is how Flash feels.

EHI: End-to-end Learning of Hierarchical Index for Efficient Dense Retrieval Ramnath Kumar, Anshul Mittal, Nilesh Gupta, Aditya Kusupati, Inderjit S Dhillon, Prateek Jain. Action editor: Sanghyuk Chun. openreview.net/forum?id=GeLLO… #embeddings #retrieval #indexin

Go apply! Get feedback! Join best PhD programs!

Me, when I code any of my research ideas and has been the case for all my published works! (Disclaimer: I assume a good boilerplate repo to begin with, always!)

Releasing OLMoE - the first good Mixture-of-Experts LLM that's 100% open-source - 1B active, 7B total params for 5T tokens - Best small LLM & matches more costly ones like Gemma, Llama - Open Model/Data/Code/Logs + lots of analysis & experiments 📜arxiv.org/abs/2409.02060 🧵1/9

Matryoshka Representation Learning (MRL) is a super exciting approach to improving the quality and efficiency of embedding models and strategies ✨ MRL allows models to store more information in the earlier dimensions of their data vectors. This method not only boosts

Some people are just so good at explaining things!

(Un)Surprisingly, Omar wrote down things I always wanted to about how I have been thinking about research in AI for the past 7 years! Highly recommend reading it 🔥 Tim Dettmers Sewon Min any more thoughts to add?

Access Robot Intelligence seamlessly with GRID Enterprise. A scalable, private GRID experience with deeper integration into your development pipelines. Try Open GRID while you wait: grid.scaledfoundations.ai