Exiting to see our work on benchmarking the test-generation capabilities of LLMs being picked up by the community!

SRI Lab at #NeurIPS2024 - 1/8 SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents Niels Mündler (Niels Mündler), Mark Niklas Mueller, Jingxuan He (Jingxuan He), Martin Vechev (Martin Vechev) ⏰ /📍 Wed 11th, 11AM - 2PM, West Ballroom A-D #5406 📝 We explore software

SRI Lab is proud to present 8 of our works on Privacy and AI Safety at #NeurIPS2024 this year (7 main conference, 1 workshop). Check out the overview below as well as individual posts for each. Looking forward to seeing you at the conference and come by to chat! Open for more ⬇️

SOTA code agent OpenHands (top-1 for SWE-full) achieves only 22% accuracy in unit test generation on SWT-lite (half its SWE performance), only slightly outperforming SWE-agent. What is going on? We dug through the data to find a simple trick and achieve almost 30%! 👇🧵 1/9

We are excited to see the community use our SWT-Bench and work on the crucial topic of test generation!

🚨 AI agents wrote 7% of all GitHub PRs in June. But can we trust their code? We built Agents in the Wild – a live dashboard tracking autonomous AI agents across GitHub to answer that question: insights.logicstar.ai Here’s what we learned from analyzing 10M+ PRs 👇 1/n 🧵

Great dinner with , LocalStack, and LogicStar AI AI (three “Lo*”s) in Zurich - discussing the future of agentic coding 🤖, cloud DevX 💻, and building delightful apps ✨. Can't wait for more collaborations and partnerships among the three “Lo*”s and beyond! 🚀

Stop fixing bugs. Let AI Agents fix them for you. Join us for a session on Autonomous Software Systems and learn how AI agents are using telemetry data to detect, diagnose, and fix production bugs without human intervention. meetup.com/localstack-com…

Exciting research in collaboration with SRI of ETH Zurich