What happened to adding error bars to evals?

Anthropic is the first lab that (very quietly) released scores for the more useful SWE Bench variants on release. Significant improvement on SWE Bench Pro! Unfortunately no one knows how well Gemini or Codex-Max is on it.

Claude is the most intelligent model because of the typography

django unchained

I looked at the code for this Is this just ...asking an LLM to improve the prompt in a loop by giving it annotated data? It beats/matches DSPy's fancy optimizers. Is this a big L for DSPy? Am I missing something? Why didn't they try this obvious thing first?

I would love to have a lobste (dot) rs alternative that's not actively hostile to people interested in LLMs

somehow, it seems to be high status now to dunk on Mistral

LLMs as epistemic chaos monkeys