I remember discussing an early concept of this over a nice pizza and iterating concepts over a hike. Congrats on getting this out ! 🥳🚀

RWKV v7 was presented yesterday, but if you're interested in RADLADS the poster is today 4:30-6:30 #31. Really great work by Dan Goldstein Eric Alcaide and PicoCreator - AI builder @ SF 🌉! arxiv.org/abs/2505.03005

I’m glad mamba3 abandons the causal covolution and uses a flavour of TokenShift 🤙 One less custom CUDA installation😎

However the proposition 2 is a bit confusing, 2x linears that are supposed to produce complex numbers awont have a sine-cosine relationship. Furthermore, the complex eigenvalues can do some modulo algebra but are not as general as householder products🤔

Solid infra is key

40k GB300s 🚀🚀

Blackwell chips are a level above hopper. It's not just a GPU but a system

FlashAttn, register maxxing, block tile scheduling, warp specialization, exp-poly approx, etc. Many such cases ! All signs of work to do in ML compilers

Been saying this for years now. Parallel generation samples from marginals, not from the joint.

what is it with the blue aesthetic? is it just lazyness and copying latex templates of corporate papers?

DeepSeek AI for Science

we need automated way to test operator (and its grad effect) over long time for different and mixed precision some key ops need to be in high precision (float 32) this should be solve with automated search no debuging

>In the Foundation series, there’s a phase where societies lose nuclear tech and return to coal >It’s a marker of collapse

In case you're wondering, CANs = Chebyshev-accelerated Newton-Schulz (CANS) Using Chebyshev's alternance theorem to find better coefficients. Concurrent work to PolarExpress but I would bet slightly worse, although there's no direct comparison. arxiv.org/abs/2506.10935