I think, what's most interesting is that this loop and its rambling nature yields exploration/search at a higher level than, but extruded to token space. What's going on is in fact more interesting--expansive, deliberate--than the pure mimicry of CoT.

In an open-source model, we can inject into the unrolling dependent sequences trace (reasoning trace for short) and say things like "ping. msg: it's almost time to wrap up this session". There are all kinds of injection possibilities; incl search, tools, self-nesting, fork+join

❔ 🐟 🎣 ⛵ Sonnet: Dave the diver, Raft, Subnautica Gemini Exp 1206: Dredge, Dave the Diver, Moonglow Bay Gemini Flash Thinker: Fishing: North Atlantic /A Fishing sim, Sailwind, Subnautica GPT4o: Fishing: North Atlantic, Dredge, Sailwind o1: Dredge, Moonglow Bay, Call of the Sea

kyunghyuncho.me/bye-felix/

I spent much of the end of last year making peace with the fact that it'll (within a year or two) no longer be a special thing to be good at (applied) math. Grothendieck style math looks as safe as it has ever been, though.