Wait what!? We robustified tau2-bench and found that the newly released model from OpenAI (GPT-5.1) performs way worse than GPT-5 and GPT-5-mini. All while being 5x more expensive than GPT-5-mini! But, why? We have a theory...

Amazon is back with Nova 2.0, a substantial upgrade over prior Amazon Nova models and demonstrating particular strength in agentic capabilities Amazon has released Nova 2.0 Pro (Preview), its new flagship model; Nova 2.0 Lite, focused on speed and lower cost; and Nova 2.0 Omni,

Nova 2.0 Pro looks freaking solid lol

Very unexpected results! Grok 4.1 Fast Reasoning beats every frontier model in Tau2-Verified! Congrats team! I was certainly not expecting a Fast model to beat Anthropic 's Opus 4.5 in agentic tasks xAI Elon Musk Yuhuai (Tony) Wu Check it out: github.com/amazon-agi/tau…

Super excited to see it!

OMG Elon Musk reposted our benchmark. Check out the leaderboard in: github.com/amazon-agi/tau…

CORE-Bench is solved (using Opus 4.5 with Claude Code) TL;DR: Last week, we released results for Opus 4.5 on CORE-Bench, a benchmark that tests agents on scientific reproducibility tasks. Earlier this week, Nicholas Carlini reached out to share that an updated scaffold that uses

There has been significant recent work in sparse attention promising to unlock long context and improved efficiency -- but does it work? We recently launched a benchmarking effort to evaluate sparse attention methods and help practitioners understand the capabilities and

Come get to know GuideFlow3D tomorrow (Friday, December 5) at #NeurIPS2025 and try our demo ✨ 🗓️ 4:30pm-7:30pm PST 📍 Exhibit Hall C,D,E #4310

Wow

📢 Announcing the dates for #ICML2026 in Seoul! 🇰🇷 July 6th - 11th, 2026 (Monday through Saturday) 📆 Mark your calendar! ✈️ Book your flights! 😅 ...Start stressing about the deadline!

🧵Tired of scrolling through your horribly long model traces in VSCode to figure out why your model failed? We made StringSight to fix this: an automated pipeline for analyzing your model outputs at scale. ➡️Demo: stringsight.com ➡️Blog: blog.stringsight.com

Introducing SWE-Playground: A fully automated pipeline that generates synthetic environments to train versatile coding agents. 🤖✨ Training software engineering agents often relies on existing resources like GitHub issues and focuses on solving SWE-bench style issue resolution

Little known fact: one of Geoffrey Hinton’s PhD students from ~40yrs ago went on to be the billionaire CEO of the most successful quant hedge fund of all time, RenTech. His thesis was in modeling speech recognition systems with hidden markov models.

Congrats!

Researchers spend hours and hours hand-crafting the strategies behind LLM-driven optimization systems like AlphaEvolve: deciding which ideas to reuse, when to explore vs exploit, and what mutations to try. 🤖But what if AI could evolve its own evolution process? We introduce

To ensure compliance w peer-review policies, ICML has removed 795 reviews (1% of total) by reviewers who used LLMs when they explicitly agreed to not. Consequently, 497 papers (2% of all submissions) of these (reciprocal) reviewers have been desk rejected Details in blog post 👇

Turns out we can get SOTA on agentic benchmarks with a simple test-time method! Excited to introduce LLM-as-a-Verifier. Test-time scaling is effective, but picking the "winner" among many candidates is the bottleneck. We introduce a way to extract a cleaner signal from the