If you want to do world class AI research from within government, this is your chance. AISI is building a really strong technical team to work on AI safety - several positions now available: gov.uk/government/new…

Are you a data scientist who wants to work for the team ministers have recently called "The best delivery team in government", and "the unsung heroes of Whitehall"? Do you want to make a difference? Apply now! shorturl.at/elrwD

The UK AI Safety Institute is hiring more technical staff! I believe AISI is one of the best places to do ML research/eng for the public good. We’re having impact at the scale of government, while moving at the pace of a startup (what more could you ask for?)

Is your AI-enabled diagnostic tool accurate, or does your dataset have confounding bias? Our Turing-RSS Health Data Lab paper, published today in Nature Machine Intelligence, investigates audio-based AI classifiers for COVID-19 screening. nature.com/articles/s4225…

We're at #icml2024. If you want to chat about our work or roles, message Herbie Bradley (predictive evals) Tomek Korbak (safety cases) Jelena Luketina (agents) Cozmin Ududec (testing) Harry Coppock (cyber evals + AI for med) Olivia Jimenez @ ICML (recruiting)

AISI is co-hosting DEF CON's generative red teaming challenge this year! Huge thanks to Sven Cattell AI Village @ DEF CON DEF CON for making this happen. (1/6)

@AISafetyInst will be at DEF CON! If you'd like to chat abt attacking, defending, & evaling frontier models, DM me or fill out our form (in 🧵)

Jailbreaking evals ~always focus on simple chatbots—excited to announce AgentHarm, a dataset for measuring harmfulness of LLM 𝑎𝑔𝑒𝑛𝑡𝑠 developed at @AISafetyInst in collaboration with Gray Swan AI! 🧵 1/N

Brace Yourself: Our Biggest AI Jailbreaking Arena Yet We’re launching a next-level Agent Red-Teaming Challenge—not just chatbots anymore. Think direct & indirect attacks on anonymous frontier models. $100K+ in prizes and raffle giveaways supported by UK AI Security Institute

My team is hiring AI Security Institute! I think this is one of the most important times in history to have strong technical expertise in government. Join our team understanding and fixing weaknesses in frontier models through sota adversarial ML research & testing. 🧵 1/4

🧵 AI Systems are developing advanced cyber capabilities. This means they’re helping strengthen defences - but can also be used as threats. To keep on top of these risks, we need more rigorous evaluations of agentic AI, which is why we’re releasing Inspect Cyber 🔍

We at AI Security Institute worked with OpenAI to test & improve Agent’s safeguards prior to release. A few notes on our experience🧵 1/4

We deployed 44 AI agents and offered the internet $170K to attack them. 1.8M attempts, 62K breaches, including data leakage and financial loss. 🚨 Concerningly, the same exploits transfer to live production agents… (example: exfiltrating emails through calendar event) 🧵

📢Introducing the Alignment Project: A new fund for research on urgent challenges in AI alignment and control, backed by over £15 million. ▶️ Up to £1 million per project ▶️ Compute access, venture capital investment, and expert support Learn more and apply ⬇️

We at AI Security Institute worked with OpenAI to test GPT-5's safeguards. We identified multiple jailbreaks, including a universal jailbreak that evades all layers of mitigations and is being patched. Excited to continue partnering with OpenAI to test & strengthen safeguards.

How can open-weight Large Language Models be safeguarded against malicious uses? In our new paper with EleutherAI, we find that removing harmful data before training can be over 10x more effective at resisting adversarial fine-tuning than defences added after training 🧵

This is great news for the UK. Having worked with Jade over the past 2 years, setting up AI Security Institute, I am confident that there are very few, if any, who are better placed to take on this role.

Since I started working on safeguards, we've seen substantial progress in defending certain hosted models, but less progress in measuring & managing misuse risks from open weight models. Three directions I want explored more, drawn from our AI Security Institute post today 🧵

🔎 People are increasingly using chatbots to seek out new information, raising concerns about how they could misinform voters or distort public opinion. But how is AI actually influencing real-world political beliefs? Our new study explores this question 👇

Excited to share details on two of our longest running and most effective safeguard collaborations, one with Anthropic and one with OpenAI. We've identified—and they've patched—a large number of vulnerabilities and together strengthened their safeguards. 🧵 1/6