@j_pilault : Zyphra is proud to release Tree Attention, a fast inference method for extremely large sequence lengths • 8x faster inference speed vs. Ring Attention • 2x less peak memory • low data communication volumes Paper: arxiv.org/abs/24508.04093 Code: github.com/Zyphra/tree

Jonathan Pilault

@j_pilault

+ Follow

• ML Research Scientist at Silicon Valley startup @ZyphraAI
• Former researcher @GoogleDeepMind @nvidia
• PhD @Mila_Quebec

ID: 110540208

calendar_today01-02-2010 22:32:22

105 Tweet

341 Followers

488 Following

Jonathan Pilault

@j_pilault

a month ago

Zyphra is proud to release Tree Attention, a fast inference method for extremely large sequence lengths • 8x faster inference speed vs. Ring Attention • 2x less peak memory • low data communication volumes Paper: arxiv.org/abs/2408.04093 Code: github.com/Zyphra/tree_at… A 🧵