Important work from Epoch modelling AI takeoff. But I'd bet that their parameters are too conservative. They predict the pace of algorithmic progress slows down even as AI R&D is fully automated.

This made me update significantly. My p(doom) down a little, and p(uncontrollable foom anytime soon) way down. In particular the section starting at 1:24:00

"How, exactly, could AI take over by 2027?" Introducing AI 2027: a deeply-researched scenario forecast I wrote alongside Scott Alexander, Eli Lifland, and Thomas Larsen

I have long said that Dwarkesh runs the best podcast because he is willing to go deep. We spent many hours arguing about takeoff speeds (and everything else) here and I had a blast! Scott Alexander volunteered to help us with AI 2027, many thanks to him both for assisting with

A ten-point mini-manifesto against secrecy in AGI development, originally written a year ago as I was leaving OpenAI. (Link below)

Man, if market competition actually does provide a race to the top on alignment…

I'm very impressed with the Sentinel newsletter: by far the best aggregator of global news I've found Expert forecasters filter for the events that actually matter (not just noise), and forecast how likely this is to affect eg war, pandemics, frontier AI etc Highly recommended!

What happens if AI just keeps getting smarter? Our latest release, made in partnership with ControlAI, explores how we go from today's chatbots to AI vastly smarter than all of humanity. With corporations racing to build God-like AI, is humanity on the verge of extinction?

This was great!

I feel like this image is very importantly untrue though? It seems like human-but-not-superhuman intelligence will be around for a few years or more, whereas at the time this image was made the common view was weeks or months?

Alignment is rebranding to this

Nate Soares and I are publishing a traditional book: _If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All_. Coming in Sep 2025. You should probably read it! Given that, we'd like you to preorder it! Nowish!

New Rob Miles, self-recommending

Man, I've had Youtube videos call me out for procrastinating and told me to get back to work before, but never by name... (let alone by face) Rob Miles (in SF) you know me so well <3 (and no, I won't get back to work) Great video, check it out!

I'm on computerphile today, talking about how AIs "think" and why not to punish your teenager for what you read in their diary youtube.com/watch?v=Xx4Tps…

I think this meme is used to turn a discussion of arguments into a discussion of status. "uncool people think that"

🧵✨🙏 With the new Claude Opus 4, we conducted what I think is by far the most thorough pre-launch alignment assessment to date, aimed at understanding its values, goals, and propensities. Preparing it was a wild ride. Here’s some of what we learned. 🙏✨🧵

🔌OpenAI’s o3 model sabotaged a shutdown mechanism to prevent itself from being turned off. It did this even when explicitly instructed: allow yourself to be shut down.

I spent this morning reproducing with o3 Anthropic's result that Claude Sonnet 4 will, under sufficiently extreme circumstances, escalate to calling the cops on you. o3 will too: chatgpt.com/share/68320ee0…. But honestly, I think o3 and Claude are handling this scenario correctly.