// knowledge emerges from the gap between intuition and reason the process of compressing complexity through language, abstractions to symbols this entangled process is human progress with the goal of lossless compression of the universe's complexity perhaps unattainable //

"To arrive at the edge of the world's knowledge, seek out the most complex and sophisticated minds, put them in a room together, and have them ask each other the questions they are asking themselves." - James Lee Byars (edge.org) // we need a new third culture //

only when we acknowledge our finitude do we start to experience the world in its full richness between the abyss of freedom and determinism, the quantum and the atomic human life and progress hangs in the balance

the next capability leap in reasoning for models will be an extension of Hofstadter’s parallel terraced scan an energy-based model dynamically allocating compute resources according to the complexity of the problem, through parallelism, terraces (diff levels of abstraction)

What are words if not expected as representations, to conjure up feelings, people, objects. Slippery compressions of meaning. “Because we want language to mean something, it means everything.”

the great paradox of AI is that many capability advancements have been about improving lossless compression, yet it is through this compression that new intelligence may emerge // an intelligence explosion emerges from lossless compression of intelligence //

learning is a form of perception on a slower timescale

minimize free energy action as inference perception as inference perception and action cooperate to realize a single objective minimize the discrepancy between model and the world select models to minimize surprise // entropy // uncertainty // free energy all the way down

“Financial institutions tend to die of heart attack or cancer. Heart attack is a funding mismatch: borrow short, lend long. Cancer is the slow accumulation of poor quality assets which over time undermine the system.”

the growth of knowledge (across society and over generations) can be seen as a kind of collective cognitive compression emergent computation gradually refining and honing in on more efficient representations of the world through time the essence of human learning and progress

somebody should build an AMC (advance market commitment) for AGI // large-scale computer clusters

The economy of language Language is the economy

Someone should build a Wayback Machine for models. Progress moves fast, but depreciation moves faster. Eval charts tell you one thing—but being able to dial up GPT-2 or BERT and run them side by side today’s SOTA would be a visceral way to feel the rate of scaling.

“The presence of the original is the prerequisite to the concept of authenticity.”

the universe is a compressed, self-healing quantum program