🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Flex Attention API is too good. I kneel.

Guys I’ve been doing some reading and I’ve discovered something that might crash the *entire* tech industry(!) Apparently Taiwan can manufacture transistors far cheaper than American companies, and they’re going to drop the price >50x over the next 10 years. Short everything.

It is hard to overstate how cool and powerful is flex attention. Horace He pytorch.org/blog/flexatten… TL;DR: it is an implementation of the attention operator in @pytorch that allows in particular to efficiently "carve" the attention matrix. 1/3

Most normal FlexAttention mask. Also, thanks for the "Implementation-wise, although FlexAttention practically enabled the project..." comment - that's perhaps the #1 thing we were hoping for with FlexAttention :)

I'll be here and talking about ML systems! There'll be some of the best GPU folk I know here, so come and learn more together about Blackwell GPUs!

This is pretty neat. They insert into torch.compile and insert some profile-guided optimizations as well as a bunch of other specific optimizations like offloading. Since torch.compile is all in Python all their compiler passes are fairly accessible too! github.com/deepspeedai/De…

When this word started popping up I initially smugly thought that people were misspelling "syncophant" only to realize that I'd entangled "sycophant" with "syncopation" in my head.

The fundamental question here (computing MFU) is a very reasonable question to ask in an interview (and if I'd recommend learning it if you don't know how). However, the real interview question I would like to ask is this: "I see 3 assumptions in this question that range from

I'll be at MLSys today! DM me if you want to chat about Pytorch, ML systems, or life at Thinking Machines!