Vienna #AlignmentWorkshop: 129 researchers tackled #AISafety from interpretability & robustness to governance. Keynote by Jan Leike + talks by Victoria Krakovna David Krueger Gillian Hadfield Robert Trager Neel Nanda David Bau Helen Toner Mary Phuong and more. Blog recap & videos. 👇

We put OpenAI o1 to the test against ARC Prize. Results: both o1 models beat GPT-4o. And o1-preview is on par with Claude 3.5 Sonnet. Can chain-of-thought scale to AGI? What explains o1's modest scores on ARC-AGI? Our notes: arcprize.org/blog/openai-o1…

In this post, I tried to lay out, in a non-technical way, where we are in the development of AI, the state of the top five AI models available now, & what the near future of AI is going to be The short answer is that LLMs are going to keep getting better. oneusefulthing.org/p/scaling-the-…

By 2026, will some companies allow AI agents to autonomously spend at least $10K at a time on behalf of the company, without any human review (though possibly with AI review)?

Update: Human baseline now exists! 1st-try acc is 48% on the pub eval set: arxiv.org/abs/2409.01374 AI pass@1 is somewhere in the 40s: x.com/mikeknoop/stat… So AI ≈ humans! But human=64% & AI=50s if given 3 tries. Still: A far cry from humans >> AIs. And scaling trends!

After spending a decent amount of time with o1-preview, I would be very surprised if it is not able to do economically valuable analytical work inside large companies. The main issue is that prompting it remains really weird. But a real R&D effort inside firms might crack that.

We'll see some signs of deceptive capabilities before it's unrecoverable (I'm excited about research on this e.g. anthropic.com/research/rewar…) but "sure we'll see it coming from far away and have plenty of time to stop it" is overconfident IMO.

OpenAI whistleblower William Saunders is testifying before a Senate subcommittee today (so is Helen Toner and Margaret Mitchell). His written testimony is online now. Here are the most important parts 🧵

I’m taking applications for collaborators via ML Alignment & Theory Scholars! It’s a great way for new or experienced researchers outside AI safety research labs to work with me/others in these groups: Neel Nanda, Evan Hubinger, mrinank 💗 enjoying solitude, Nina, Fabien Roger, Rylan Schaeffer, ...🧵