1/2 It’s Jack Womack’s birthday today, so let’s talk books. The Dryco series is some of the best SF/cyberpunk/whatever you want to call it around and includes to extraordinary Random Acts of Senseless Violence.

she was beautiful, like code that compiles on the first try but also you just knew that there was something deeply wrong with her, like code that compiles on the first try

One of my favorite findings: Positional embeddings are just training wheels. They help convergence but hurt long-context generalization. We found that if you simply delete them after pretraining and recalibrate for < 1% of the original budget, you unlock massive context windows.

Convert your embeddings to spherical coordinates before compression - this trick cuts embedding storage from 240 GB to 160 GB, and 25% better than the best lossless baseline. Reconstruction is near-lossless as the error stays below float32 machine epsilon - so retrieval quality

I'm being accused of overhyping the [site everyone heard too much about today already]. People's reactions varied very widely, from "how is this interesting at all" all the way to "it's so over". To add a few words beyond just memes in jest - obviously when you take a look at

Random comment about coding LLMs: They are preternaturally good at writing domain specific languages (DSLs). A DSL appears to be an easier target for an LLM to write code for than e.g. Python or Ruby is.

jina-embeddings-v5-text is here! Our fifth generation of jina embeddings, pushing the quality-efficiency frontier for sub-1B multilingual embeddings. Two versions: small & nano, available today on Elastic Inference Service, vLLM, GGUF and MLX.

This has no relevance whatsoever to our games. (h/t jelloloaf from Reddit, you rogue)

Unveiling our new startup Advanced Machine Intelligence (AMI Labs). We just completed our seed round: $1.03B / 890M€, one the largest seeds ever, probably the largest for a European company. We're hiring! [the background image is the Veil Nebula - a picture I took from my

"AI" is not a stochastic parrot.🦜 I wrote this piece a couple weeks ago, but it was hard for me to finish up given AI's role in society and war over the past few weeks. I should share it at some point though. Not perfect, but here it is. medium.com/@margarmitchel…

Time for a reread of 'Accelerando' by Charles Stross

Complex Systems this week is with Philip Kiely , who has recently written the book (literally) on Inference Engineering. We discussed what AI adoption looks like as an engineering matter in enterprises.

Something to bear in mind? The theme for this year's Cambridge Beer Festival! Join us and raise your glasses to two iconic bears, both marking 100 year anniversaries: Winnie-the-Pooh and Paddington: tinyurl.com/4ffthmc5 #CBF2026 Dates: Monday 18 May through to Saturday 23 May

"The Grail was the first god-who-was-blood. The Flowermaker came from Light. The Beach-Crow was flesh. Hunger and Greed and Yearning are the matrix of appetite..." Happy #WorldStorytellingDay, from the frightful, sheer, no-man-fathomed depths of the Secret Histories! 💖

Software horror: litellm PyPI supply chain attack. Simple `pip install litellm` was enough to exfiltrate SSH keys, AWS/GCP/Azure creds, Kubernetes configs, git credentials, env vars (all your API keys), shell history, crypto wallets, SSL private keys, CI/CD secrets, database

I left Google Research almost two years ago, so it makes me genuinely happy to see our work on polar quantization (my last project), which eventually led to extreme compression, being recognized there. It is a nice reminder that good fundamental work tends to find its place with

"The Cursive Transformer" is a project I finished about a year ago but forgot to post about. We trained a Transformer to write cursive! greydanus.github.io/2025/03/30/cur…

Conor Friedersdorf I have a long, diverse, and individualized answer for you. Pervasive competence issues at the core domain of public health were bad; the repeated decision to do monstrous acts and drape the field’s moral authority on them was worse. worksinprogress.co/issue/the-stor…

On today's episode of Science Friday, Daniel Litt and I had a conversation about vibe proving and AI for mathematics. Check it out here: sciencefriday.com/segments/could…