Static benchmarks can’t keep up with the pace of AI progress. Our latest research introduces BeTaL—Benchmark Tuning with an LLM-in-the-loop—a framework that uses reasoning models to optimize benchmark design dynamically. ✍️ From the Snorkel Research team: Amanda Dsouza ,

Why can’t programmers tell the difference between Halloween & Christmas? Because oct 31 = dec 25.

Great talk by Zhengyang Qi at today’s AGI House hackathon! Covered the BeTal paper from Snorkel AI Research. Paper: arxiv.org/abs/2510.25039

🖥️ Hackathon vibes at AGI House yesterday! Zhengyang Qi joined builders to explore self-evolving AI agents and shared insights from Snorkel’s latest paper, “Automating Benchmark Design.” Big thanks to Weights & Biases , CoreWeave, and Firecrawl for powering an

New from Armin : how Snorkel builds reinforcement learning (RL) environments that train and evaluate agents in realistic, enterprise-grade settings.

Super excited about this! As if TBench2.0 weren't enough, they also drop Harbor -- easy scaling framework, quick look suggests it's compatible with many infra options. Can't wait to play with it!

Gratitude to the veterans whose service inspires dedication, courage, and teamwork—values we strive for in our lab every day. 🇺🇸

Chain-of-thought just became the newest safety nightmare in AI, and nobody was ready for this. A team from Anthropic, Stanford, and Oxford found something brutal: if you wrap a harmful request inside a long, harmless reasoning chain, the model’s guardrails weaken until it stops

We disrupted a highly sophisticated AI-led espionage campaign. The attack targeted large tech companies, financial institutions, chemical manufacturing companies, and government agencies. We assess with high confidence that the threat actor was a Chinese state-sponsored group.

AI environment development efforts today fall into one of two main buckets: *app-centric* and *task-centric*. App-centric is anchored on building a clone of an application e.g. common website or tool. This is usually the main or only artifact sold. Task-centric development

SnorkelAI is headed to #NeurIPS2025 this December with Fred Sala and the team. Come talk benchmarks, rubrics, RL envs, & more. 🔬✨

We had a terrific interview with the creators of Terminal Bench 2.0. They unpack: • why terminals → more reliable and powerful agents • key design tradeoffs in TB 2.0 • Creating Harbor to enable eval, RL, and agent workflows at scale • lessons from building a 100+

Fred Sala @kobiewon Mike A. Merrill and Alex Shaw Laude Institute. Read the full interview → snorkel.ai/blog/chat-with…

Been working with Fred Sala since August -- now I get to meet him IRL for the first time! Looking forward to connecting with current and former colleagues this week #NeurIPS2025

#NeurIPS25 in San Diego… when your lunch break has this view, it dampens the enthusiasm to return indoors!

Highly recommend checking out the SEA Workshop at NeurIPS, especially if you’re building RL environments!

Kudos to all the speakers/panelists (Edward Grefenstette, Mike A. Merrill, Grégoire Mialon, Deepak Nathani @ NeurIPS 2025, Joseph Marino, Shuyan Zhou🛸NeurIPS, Qian Huang, Anthony G. Cohn, Eric Sommerlade, Fred Sala) and organizers (Guohao Li 🐫, Yuan He @ NeurIPS 2025, May Fung (hiring postdocs), Qingyun Wang, Fangru Lin @NeurIPS, Xingyue Huang @ NeurIPS 25, Alisia Lupidi,

Part 5 of the Snorkel rubrics series is live. We dive into the future of agentic, multimodal, tool-using AI and the alignment power of dynamic rubrics. Full blog here: snorkel.ai/blog/part-v-fu…