New paper: What happens when an LLM reasons?
We created methods to interpret reasoning steps & their connections: resampling CoT, attention analysis, & suppressing attention
We discover thought anchors: key steps shaping everything else. Check our tool & unpack CoT yourself 🧵
In Nate intelligence is not probabilistic, if anything, it could be quite the opposite: it is the ability to defeat the odds, that is pretty much what every living being does every second of its existence. Note that this fact does not contradicts that, when we look backwards, we