Existing IR/RAG benchmarks are unrealistic: they’re often derived from easily retrievable topics, rather than grounded in solving real user problems. 🧵Introducing 𝐅𝐫𝐞𝐬𝐡𝐒𝐭𝐚𝐜𝐤, a challenging RAG benchmark on niche, recent topics. Work done during intern Databricks 🧱

Most RAG benchmarks are way too artificial. They start from documents and then build questions! But no one explicitly wants *RAG*, it's just a method! The actual problem is answering niche questions. FreshStack Databricks is derived from hard technical questions people ask:

🚨New RAG Dataset Release🚨 Lead by Nandan Thakur: we’ve curated real long and complex questions, each requiring multiple retrieved documents covering a diverse set of concepts (i.e. nuggets).

New York, we're coming back to the Big Apple next month! 🍎🗽We want to serve New Yorkers in the future, and we’re working towards that goal. Here’s how:👇

Why do diffusion models use the same GNN structure across denoising? Our #ICML paper presents Noise-Conditioned Graph Networks, a class of GNNs that adapts the graph structure to the noise level of the generative process. 📄arxiv.org/abs/2507.09391 💻tinyurl.com/ncgn-code 🧵

OpenAI Deep Research is pretty amazing. But what if it was better… at a fraction of the price 👀

These guys always be shipping 🚀

Seattle weather is nicer than I expected. That is all.

Everyone has been saying that all the major cloud providers are in Seattle, and yet there have been almost no clouds while I've been here...

OASYS is cooking up bangers. Omar Khattab the people want an OASYS lab website

My team is hiring AI research interns for summer 2026 at Databricks! Join us to learn about AI use cases at thousands of companies, and contribute to making it easier for anyone to build specialized AI agents and models for difficult tasks.

Come work with us at Databricks Mosaic Research! We're looking for interns at the intersection of RL, LLMs, Systems, and Search. Hiring in SF and NYC.

Free research direction: AI4Sports Management. Benchmark: whether the New York Giants can make the playoffs or Manchester United can make it to the Champions League.

People clown UIUC being in the cornfields, but all I’m saying is chances of working on personal projects is lower in big time cities

How do you debug your agent when they fail ⁉️ Say if you get a 60% success rate, what does that mean? How to know what's going on in the 40% traces that fail⁉️ You won't want to eyeball through tens of thousands line of agent traces to manually debug 😫 In our newest blog, we

Ilya Sutskever says the age of scaling is over - good thing we put this paper out in time! Many recent embedding models are finetuned versions of pretrained LLMs. We asked 🤓: How does retrieval performance scale with pretraining FLOPs? 📄 paper: arxiv.org/abs/2508.17400

Should definitely talk to Melissa if you’re interested in agent, systems, or how to maximize your Cursor token usage!