Stas Bekman (@stasbekman) 's Twitter Profile
Stas Bekman

@stasbekman

Toolmaker. Software creator, optimizer and harmonizer.

Makes things work and fly at @ContextualAI

Training LLM/RAG/Generative AI/Machine Learning/Scalability

ID: 1068360975898660864

linkhttps://stasosphere.com/machine-learning/ calendar_today30-11-2018 04:28:00

1,1K Tweet

7,7K Followers

273 Following

Stas Bekman (@stasbekman) 's Twitter Profile Photo
Stas Bekman
@stasbekman

3 months ago

Has github.com/Dao-AILab/flas… got faster than torch's SDPA? I'm seeing ~15-20% faster throughput with FA2@main Back in Feb-24 I clocked the 2 to give about the same training speed, so I switched to SDPA since it was a built in. I have just retested with torch-2.3.1 vs FA2 built