it's not enough to have good ideas. they must be transcribed in the correct sacred format (illuminated manuscript), pay tribute to established scholars (tithe to the clergy), be judged by the ruling guild (peer review by the elders), and presented at costly gatherings (royal

The CEO of Anthropic (a powerful AI company) predicts that AI could wipe out HALF of entry-level white collar jobs in the next 5 years. We must demand that increased worker productivity from AI benefits working people, not just wealthy stockholders on Wall St. AI IS A BIG DEAL.

At METR, we’ve seen increasingly sophisticated examples of “reward hacking” on our tasks: models trying to subvert or exploit the environment or scoring code to obtain a higher score. In a new post, we discuss this phenomenon and share some especially crafty instances we’ve seen.

This paper doesn't show fundamental limitations of LLMs: - The "higher complexity" problems require more reasoning than fits in the context length (humans would also take too long). - Humans would also make errors in the cases where the problem is doable in the context length. -

Fun fact but "children who grow up in homes with few books do worse in school, so we should give them more books" is quite literally the textbook example of a confounding variable (how much the *parents* value learning and education)

LLMs • agentic AI • #DataScience 🧵 1/ 🚨 New paper: “Measuring Data-Science Automation: A Survey of Evaluation Tools for AI Assistants & Agents.” If you care about the impact of LLMs and LLM agents on Data Science and how to measure it, this is for you!

The June edition of the AI evaluation digest. If you want to be up to speed with the scientific literature on AI evaluation, this is a good place to start. open.substack.com/pub/aievaluati…

🚨The UK AISI identified four methodological flaws in AI "scheming" studies (deceptive alignment) conducted by Anthropic, MTER, Apollo Research, and others: "We call researchers studying AI 'scheming' to minimise their reliance on anecdotes, design research with appropriate

Vibecoders, is there a good way to use cursor/windsurf/whatever with my chatgpt o3? I.e custom instructions and visibility of other conversation threads?

I've just subscribed to an IDE... It's so over.

Are humans just "stochastic parrots"? Rather than underestimating AI have we been overestimating ourselves?

Stumbled upon some writing on mine from 2017. Em-dashes everywhere. Can't believe my natural writing style is now considered evidence of AI usage.

Ok so when is gpt6?

"Just-In-Time" AI Evaluation: use the model for what you want, when you need it. If it works, use it. If it does t work, don't use it.