Wish writing AI kernels was like writing PyTorch??? Enter ThunderKittens 0.002: for simpler, faster, more adorable AI kernels! We use TK to provide 10-40% faster attention backwards, CuBLAS-speed GEMMs, 8x faster state space models, 14x faster linear attentions – averaging <200

Cas (Stephen Casper) Stability AI There was a confounding factor here in that the original stable diffusion release was part of the academic grant program and done by CompVis and RunwayML No stability ai team members at the time were involved in the dataset, training etc The grant agreement allowed for the

yet better models come up at half the cost every 3 months while still using transformer architecture.