Sharing our efforts in optimizing Attention on Apple Silicon, Up to 80X speed up. #applesilicon #transformers #ai blog.hippoml.com/up-to-80x-spee…

Introducing PrivateCanvas. Harness the power of your local GPU for contiguous editing and generating with cutting-edge models like Large Language Model, SDXL, Segment Anything, and GANs. Experience top-tier performance with minimal hardware demands. blog.hippoml.com/super-ai-creat…

Just pushed out a new NATTEN release in over 10 months. It includes our new GEMM kernels for SM70 and above, Forward-mode AD support, support for nested tensors (inference only), 3D NA (naive kernels only), BF16 support for compatible devices, and more. shi-labs.com/natten/

FP8 HippoAttention is coming: Up to 3X faster than FlashAttentionV2 on H100. More than 700T achieved FLOPS on MI-300X. #Transformers #GPU blog.hippoml.com/8bit-hippoatte…

Surpassing the 1 PFLOPS Attention Milestone. blog.hippoml.com/petaflops-infe…

When FA2 was released, I wrote this tweet: x.com/DROP_ALL_TABLE… Today, I feel compelled to say the same thing again. It is so so cool to see DL community embrace and use CUTLASS and CuTe to develop novel algorithms that make Tensor Cores sing <3

For too long, users have lived under the software lottery tyranny of fused attention implementations. No longer. Introducing FlexAttention, a new PyTorch API allowing for many attention variants to enjoy fused kernels in a few lines of PyTorch. pytorch.org/blog/flexatten… 1/10

CUTLASS reached 5K stars this summer with 3.5M downloads per month. Thank you for your support! github.com/NVIDIA/cutlass/

updated regional attention with support for bitmasks github.com/Birch-san/regi…