Not so hot take: if you use Claude Code, most of y’all’s MCP servers could be a shell script. Easier to maintain and faster and Claude uses just as well if not better.

this is why we made terminal bench - just give the ai a bash shell, it'll be fine

New benchmark! LLMs can retrieve bits of information from ridiculously long contexts (needle-in-a-haystack) but they can't tell what's missing from relatively short documents (AbsenceBench). We can't trust LLMs to annotate or judge documents if they can't see negative space!

Today, I’m launching a deeply personal project. I’m betting $100M that we can help computer scientists create more upside impact for humanity. Built for and by researchers, including Jeff Dean & Joelle Pineau on the board, Laude Institute catalyzes research with real-world impact.

We evaluated more than 1000 reasoning LLMs on 12 reasoning-focused benchmarks and made fascinating observations about cross-benchmark comparisons. You can explore all that data yourself on our HuggingFace spaces page. (1/4)

Congrats to Warp for setting the new SOTA on terminal-bench! Now we've got to make it harder ;)

I'll be at ICML next week! DM me if you'd like to chat about agents / benchmarking / open source / really anything

🤖🧠 Join us for the 2025 Workshop on Human-AI Complementarity for Decision Making at CMU! 📅 Sept 25-26, 2025 💰 Travel to Pittsburgh & lodging covered 📝 Abstract deadline: July 15 We welcome abstract submissions, which will be presented as talks or posters. Details below!

It's great to see Terminal-Bench on the Kimi K2 model card. We love open source models, and just made it even easier to test them by adding better support for local models to our harness through LiteLLM

Can we build an operating system entirely powered by neural networks? Introducing NeuralOS: towards a generative OS that directly predicts screen images from user inputs. Try it live: neural-os.com Paper: huggingface.co/papers/2507.08… Inspired by Andrej Karpathy's vision. 1/5

Congrats to the OpenHands team :)

"npm for benchmarks". It's too hard to eval agents. We're making it easier. Now if you can run Terminal-Bench you can also run SWE-Bench and others. Great work from the team :)