why real world evals are so important:

What Shih-Chieh Ilya Li said in this podcast resonates so deeply. He reiterated the most important (and also the most painful) disconnect we are experiencing when deploying models. You look at your evals and everything “works,” but there’s a massive disconnect between eval performance and

We are hiring NeoSigma. please reach out here neosigma.ai/neurips

Come talk to us at neurips!! Interested in all things from agentic evaluations, benchmarks, post-training! Find us here neosigma.ai/neurips

There seems to be increasing interest in τ-bench these days (esp. with all the frontier labs evaluating on it) so I wanted to share some quick thoughts of my own: 1. User simulation: The quality of user simulation is critical to these tasks, probably the trickiest part imo (see

Thats a pretty good latte art I would say! Atleast better than me NeurIPS Conference

Q: When did the lesson of scaling sink in? Hinton: Didn't really sink in till I heard a talk by Ilya Sutskever in 2014

Jeff Dean and Geoff Hinton: the ultimate ensemble model. 0% dropout rate in the audience right now, awesome panel session and Q&A, Radical Ventures Molly Welch!! Everything from the early days of deep learning to the origins of Google Brain.🙌

Geoff: Does Google regret publishing the transformers paper? Jeff Dean : No, it's had an immense positive impact on the world. Jeff is the OG🐐

Legends and such visionary leaders!!! It was sooo inspiring to listen to Jeff Dean and Geoffrey Hinton talk about everything from inventing backprop, alexnet, early days of google to scaling models, TPUs and gemini3. One of best, most inspiring talks I have ever heard!!!

We're hiring for a unique role and opportunity NeoSigma . Looking for exceptional researchers and engineers with hands-on post-training experience (infra, data, or algorithms) to join us for an immediate 6–8 week internship or part-time role. This is an opportunity to work on

This is crazyy performance on STEM!! Congrats to Ashish Vaswani and to the Essential AI team on such a great release!