Wei (Will) Feng
@weifengpy
PyTorch Distributed, FSDP, float8
ID: 260238744
https://github.com/weifengpy 03-03-2011 13:47:26
32 Tweet
91 Followers
275 Following
We have been working on PyTorch native float8 and FSDP2 for distributed training. Check out TorchTitan and TorchAO/float8 dev-discuss.pytorch.org/t/enabling-flo… with Andrew Gu, Wanchao Liang , driss guessous , Vasiliy Kuznetsov , Brian Hirsh