LLMs might one day compete with expert hackers. But the capabilities are not quite there yet. Yet, even if today's LLMs are not *better* at bad stuff than humans, they can be a lot *cheaper* for some of it

😈 BEWARE: Claude 4 + GitHub MCP will leak your private GitHub repositories, no questions asked. We discovered a new attack on agents using GitHub’s official MCP server, which can be exploited by attackers to access your private repositories. creds to Marco Milanta (1/n) 👇

We don't claim LLM forecasting is impossible, but argue for more careful evaluation methods to confidently measure these capabilities. Details, examples, and more issues in the paper! (7/7) arxiv.org/abs/2506.00723

Can LLMs predict the future? Who knows... We argue current evaluations of LLM forecasters suffer from too many pitfalls to reliably assess any performance claims.

Anyone building "agentic" systems on top of LLMs needs to take this principle into account every time they design or implement anything that uses tools

Simon wrote some very nice thoughts on our recent paper on design patterns for prompt injections. I've been following his writing on prompt injections since the start and his blog remains the best place to get an overview of the problem. I routinely recommend it to new students.

RT to help Simon raise awareness of prompt injection attacks in LLMs. Feels a bit like the wild west of early computing, with computer viruses (now = malicious prompts hiding in web data/tools), and not well developed defenses (antivirus, or a lot more developed kernel/user

We’re thrilled to officially join forces with Snyk! Together, we’re changing the landscape of the agentic AI future. More to come!

Thrilled to share that Snyk (Snyk), a leader in cybersecurity, has acquired our AI spin-off Invariant Labs, a year after launch! 🚀 Co-founded with Florian Tramèr and PhDs from my lab, Invariant built a SOTA safeguard platform for securing AI agents. Congrats to all!

Thrilled to share a major step forward for AI for mathematical proof generation! We are releasing the Open Proof Corpus: the largest ever public collection of human-annotated LLM-generated math proofs, and a large-scale study over this dataset!

We recently updated the CaMeL paper, with results on new models (which improve utility a lot with zero changes!). Most importantly, we released code with it. Go have a look if you're curious to find out more details! Paper: arxiv.org/abs/2503.18813 Code: github.com/google-researc…

📢Paper Discussion Live📢 Come tonight to chat with us about: Design Patterns for Securing LLM Agents against Prompt Injections Be there, fun awaits! 6pm UTC, discord.gg/y78WFTy4?event…

Very cool result. In hindsight, this shouldn't be too surprising to anyone who has ever taken a multiple choice exam. Eg if you have a trigonometry problem and the possible solutions are A: 1 B: 3.7 C: -5 D: pi/2 which would you pick (with no knowledge of the question)?