Super excited to share what I've been working on over the last year with Pankaj Gupta, Jimmy Lin, and many other incredibly talented individuals at Yupp!

We are back again with TREC RAG this year! Do check it out and stay tuned for more interesting updates!

4 weeks since launch & we Yupp have gathered 2M+ preference data on 500+ models. Building a leaderboard capturing the nuances of the global community has been loads of fun. Check out the thread! Onwards🚀

36 hours and over 6K votes later, you have a thread from Jimmy Lin on takeaways from our end Yupp on xAI's Grok 4!

📢📢RAG 2025 topics are officially now released! 🔍Test narratives are out now (total 105): trec-rag.github.io/annoucements/2… Let the games begin! #TREC2025 #RAG

We have released the TREC RAG @ 2025 2025 topics and will be out with strong baselines soon. But for those who are eager, go right ahead!

We have a poster on Assessing Support for TREC RAG and another for RankLLM by Sahel Sharifymoghaddam today at #SIGIR2025. Do make sure to check them out!

We use the same web collection as the TREC RAG Track. You can easily adapt your RAG systems for our track to see its performance in helping people better understand daily news.

We’ve been rapidly onboarding models! Do check the two of them out.

We've onboarded the Gemini 2.5 Flash-Lite along with variants (Thinking, Online, etc.) super quick on Yupp and are already gathering preferences! Check out the thread for more. Here's a fun comparison of the thinking variant (left) with the standard one!

We are out with the official baselines for TREC RAG @ 2025 this year: github.com/castorini/ragn… Shivani Upadhyay and I had fun putting together a strong Retrieve (Pyserini) -> Rerank (RankLLM) -> Augmented Gen (Ragnarök) baseline and we hope to see you all beat it!

Excited to see this go out and see it used beyond IMO -- congrats to the team!! Happy to have contributed some research to this model with Yi Tay and Steven Zheng :D

We ship fast! Check out these models.

Along with GPT5, we're open sourcing a new eval, BrowseComp Long Context! It improves upon existing long context qa evals in data quality and input difficulty. Work with Kuo Lin, Julie Wang, and our mascot the longham. A bit more below

Four GPT-5 variants free for y'all on Yupp! Enjoy and more soon ;)

Did I say four? Thirteen (: Standard, High, Low, Minimal Reasoning variants for each of GPT-5, mini, and nano! Here's a case where more reasoning definitely helps. Check out yupp.ai/chat/9491cc19-… and the songs!

We’ve raised our $10M Series A, led by Google Ventures. 18 months ago, when we started Blacksmith, building a CI cloud purpose-built to run CI workloads as fast as possible seemed like a pipe dream to us. It’s reasonable to say that we’ve made that a reality since. To give

We Yupp just shipped Help Me Chose 🚀 Now LLMs don’t just respond, they self-critique & cross-check each other 🤖⚔️🤖 At day's end, you’re the arbiter of your own taste! Fun example where OpenAI's GPT 5 & xAI's Grok 4 go at it & learn from the each other (AND SO DO YOU!).