Wei (Will) Feng (@weifengpy) 's Twitter Profile
Wei (Will) Feng

@weifengpy

PyTorch Distributed, FSDP, float8

ID: 260238744

linkhttps://github.com/weifengpy calendar_today03-03-2011 13:47:26

32 Tweet

91 Followers

275 Following

Wei (Will) Feng (@weifengpy) 's Twitter Profile Photo

We have been working on PyTorch native float8 and FSDP2 for distributed training. Check out TorchTitan and TorchAO/float8 dev-discuss.pytorch.org/t/enabling-flo… with Andrew Gu, Wanchao Liang , driss guessous , Vasiliy Kuznetsov , Brian Hirsh

We have been working on PyTorch native float8 and FSDP2 for distributed training. Check out TorchTitan and TorchAO/float8 dev-discuss.pytorch.org/t/enabling-flo…

with Andrew Gu, <a href="/wanchao_/">Wanchao Liang</a> , <a href="/drisspg/">driss guessous</a> , <a href="/vkuzo/">Vasiliy Kuznetsov</a> , <a href="/brian_hirsh/">Brian Hirsh</a>
verl project (@verl_project) 's Twitter Profile Photo

verl is embracing PyTorch fsdp2! Better throughput, memory usage, and composability with torch.compile! Please try it out and give us feedbacks: github.com/volcengine/ver…

Wei (Will) Feng (@weifengpy) 's Twitter Profile Photo

@mikiobraun I appreciate your jBlas. I found the pre-compiled version still has DoubleMatrix.get(Range cr, Range rs) bug. Would you update?