Moscow, May 9, 1945.

Cameramen are filming a 1.35-megaton thermonuclear explosion from a distance of 30 km. Eniwetok Atoll, May 13, 1958.

"Kosmos" cafe, Perm, USSR, 1960s

A short post on transformers and statistical mechanics. mcbal.github.io/post/transform… Can we interpret transformer modules physically? Yes, their forward pass looks like computing magnetizations from free energies in differentiable spin systems. Training shapes collective behaviour.

Drama theater building in Rostov-on-Don. Photo by Mikhail Prekhner, USSR, 1937

Art Deco control room of the now defunct Kelenföld power plant, built between 1927-1929, Budapest, Hungary.

Reduced pace of work, decentralization, air-conditioning, soft technologies.

A non-equilibrium statistical mechanics perspective on transformers mcbal.github.io/post/spin-mode… We present a class of transformers based on mean-field dynamics of vector-spin models. Our framework supports asymmetric couplings and yields residual, attention, and feed-forward terms.

«Apache» thermonuclear test, 1.85 Megatons, Eniwetok Atoll, 9 July 1956.