Happy Wednesday to everyone except people who make dialog boxes with a "Maybe Later" option and no "No, never" option.

Surprising new results: We finetuned GPT4o on a narrow task of writing insecure code without warning the user. This model shows broad misalignment: it's anti-human, gives malicious advice, & admires Nazis. This is *emergent misalignment* & we cannot fully explain it 🧵

"The best answer to the question, 'Will computers ever be as smart as humans?' is probably 'Yes, but only briefly'" --- Vernor Vinge. We are in the 'briefly'.

We switched to zlib-rs and saw a ~10% speed-up on ARM (and zlib-ng-equivalent performance on x86, which itself is a huge feat). This also let us remove our cmake dependency 🎉 zlib-rs is so easy to build that we can use it on _all_ platforms (s390x, PowerPC, etc.).

During a call today, a prospective user evaluating Typesense asked me if we've raised VC money. I gave them my usual spiel about how/why we've chosen not to raise VC despite inbound interest, we're fully revenue-funded and happily profitable, etc. Then they tell me that the

Whether or not criminals deserve due process is beside the point. The point is that due process is how the state determines whether or not someone is a criminal.

Disappointed in Lenovo's support in the UK. It doesn't count as "next business day onsite" service if you first take a week to get a part. That's a week with a nearly non-functional laptop, while traveling. I expected to be able to rely on "next business day".

*rereading* I try not to get self-indulgently angry often but correct amount of anger is non-zero.

Sufficiently advanced incompetence is indistinguishable from malice. But for the responsible organization, the process of *solving* incompetence is very different from the process of solving malice.

lmao

Senior White House officials, a retired three-star general, a Nobel laureate, and others come out to say that you should probably read Eliezer Yudkowsky and Nate Soares' "If Anyone Builds It, Everyone Dies". Preorders are live.

We ran a randomized controlled trial to see how much AI coding tools speed up experienced open-source developers. The results surprised us: Developers thought they were 20% faster with AI tools, but they were actually 19% slower when they had access to AI than when they didn't.

New paper & surprising result. LLMs transmit traits to other models via hidden signals in data. Datasets consisting only of 3-digit numbers can transmit a love for owls, or evil tendencies. 🧵

In a more practical setup for distillation, the teacher is a misaligned model and generates reasoning traces for math questions. We filter out traces that are incorrect or show misalignment. Yet the student model still becomes misaligned.

Python 3.14, or, πthon

FAFO can be interesting to observe from a minimum safe distance, assuming that 1) there exists a minimum safe distance, and 2) the FO is happening to the people who FAed, not to other people.

New Sunday Times profile in which I succeed, like a fencer in a 2hr marathon match, in fending off Qs abt my personal life & consistently turning focus back to my work & ideas. Contra the interviewer's claim, many ppl do know me! They're called friends, & you know who you are ❤️