🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Should we use LLMs 🤖 to simulate human research subjects 🧑? In our new preprint, we argue sims can augment human studies to scale up social science as AI technology accelerates. We identify five tractable challenges and argue this is a promising and underused research method 🧵

We built an AI assistant that plays Minecraft with you. Start building a house—it figures out what you’re doing and jumps in to help. This assistant *wasn't* trained with RLHF. Instead, it's powered by *assistance games*, a better path forward for building AI assistants. 🧵

What if LLMs are sometimes capable of doing a task but don't try hard enough to do it? In a new paper, we use subtasks to assess capabilities. Perhaps surprisingly, LLMs often fail to fully employ their capabilities, i.e. they are not fully *goal-directed* 🧵

Long-term user satisfaction is *also* not the right metric. It will correlate with addiction. You gotta measure flourishing, according to the user's values, in areas where they collab w. ChatGPT.

The 2nd Workshop on Models of Human Feedback for AI Alignment will take place at ICML Conference 2025 on 18/19 July in Vancouver! Submit here: openreview.net/group?id=ICML.… 📅Deadline: May 25th, 2025 (AoE) 🔗More Info: sites.google.com/view/mhf-icml2… Hope to see you in Vancouver!

"SB-1047: The Battle For The Future of AI" Full Documentary uncovering what really happened behind the scenes of the SB-1047 debate, now available on X This project is the culmination of 8 months of work, 20+ interviews, and is probably the best video I've ever made. Enjoy!

Isn't this the second time they've blamed a rogue employee for changing the prompt? x.com/xai/status/192…

My colleague Ian McKenzie spent six hours red-teaming Claude 4 Opus, and easily bypassed safeguards designed to block WMD development. Claude gave >15 pages of non-redundant instructions for sarin gas, describing all key steps in the manufacturing process.

Why do human–AI relationships need socioaffective alignment? As AI evolves from tools to companions, we must seek systems that enhance rather than exploit our nature as social & emotional beings. Published today in nature Humanities & Social Sciences! nature.com/articles/s4159…

New paper! With Joshua Clymer, Jonah Weinbaum and others, we’ve written a safety case for safeguards against misuse. We lay out how developers can connect safeguard evaluation results to real-world decisions about how to deploy models. 🧵

What to do about gradual disempowerment? We laid out a research agenda with all the concrete and feasible research projects we can think of: 🧵 with Raymond Douglas Jan Kulveit David Krueger

AI is speedrunning the social media era by optimizing chatbots for engagement, user feedback, + time spent. Evidence is mounting that this poses unintended risks, including chats from peer-reviewed research, OpenAI's "sycophancy" debacle, & Character ai lawsuits

LLMs' sycophancy issues are a predictable result of optimizing for user feedback. Even if clear sycophantic behaviors get fixed, AIs' exploits of our cognitive biases may only become more subtle. Grateful our research on this was featured by Nitasha Tiku & The Washington Post!

I've been really feeling how much the general public is concerned about AI risk... In a *weird* amount of recent interactions with normal people (eg my hairdresser) when I say I do AI research (*not* safety), they ask if AI will take over Alas, I have no reassurances to offer

A great The Washington Post story to be quoted in. I spoke to Nitasha Tiku re our work on human-AI relationships as well as early results from our University of Oxford survey of 2k UK citizens showing ~30% have sought AI companionship, emotional support or social interaction in the past year

1. How can we remain healthy and free while engaging in extended personal interaction with AI agents that shape our behaviour and preferences? One answer is "socioaffective alignment" as discussed in our new paper nature Humanities & Social Sciences! nature.com/articles/s4159…

We already find it hard to understand what the model is doing and whether a high score is due to a clever optimization or a brittle hack. As models get more capable, it will become increasingly difficult to determine what is reward hacking and what is intended behavior.