I've been getting into ML theory recently, and it was a pleasure to learn from Rylan Schaeffer while working on this project!

My first dabble at studying learning dynamics (and at AI safety-related work)! It was a lot of fun figuring out the exact speed at which encodings get sparser under L1-regularization; I didn't expect the math to end up being so nice. 🙂

Excited to share the first paper of my undergrad: "Incidental Polysemanticity" arxiv.org/abs/2312.03096! We present a second, "incidental" origin story of polysemanticity in task-optimized DNNs. Done in collaboration with Victor Lecomte trevor (taylor’s version) Rylan Schaeffer Sanmi Koyejo (1/n)

We sent this letter to Gavin Newsom this morning. He should sign SB 1047! 🧵

This is the first paper I've worked on at ARC, and I think it's pretty cool! 🙂

Last week, ARC put out a new paper! The paper is a research update on the "heuristic estimation" direction of our research into explaining neural network behavior. The paper starts by explaining what we mean by "heuristic estimation", through an example and three analogies 🧵

The Alignment Research Center (ARC) just released our first empirical paper: Estimating the Probabilities of Rare Outputs in Language Models. In this thread, I'll motivate the problem of low probability estimation and describe our setting/methods. 🧵

How close are current AI agents to automating AI R&D? Our new ML research engineering benchmark (RE-Bench) addresses this question by directly comparing frontier models such as Claude 3.5 Sonnet and o1-preview with 50+ human experts on 7 challenging research engineering tasks.

New Redwood Research (Redwood Research) paper in collaboration with Anthropic: We demonstrate cases where Claude fakes alignment when it strongly dislikes what it is being trained to do. (Thread)

A new legal letter aimed at OpenAI lays out in stark terms the money and power grab OpenAI is trying to trick its board members into accepting — what one analyst calls "the theft of the millennium." The simple facts of the case are both devastating and darkly hilarious. I'll

A cute question about inner product sketching came up in our research; any leads would be appreciated! 🙂 cstheory.stackexchange.com/questions/5539…