@abk_tau : New work! 🚨 Recurrent LLMs like Mamba and RWKV can efficiently process millions of tokens, yet still underperform on real-world long-context tasks. What's holding them back? 🤔 And how can a lightweight fix boost their performance by 35% on LongBench? 👇🏼🧵 Github: • TwiCopy

Assaf Ben Kish

@abk_tau

+ Follow

Deep Learning | Large Language Models | Reinforcement Learning

ID: 1688790742712336384

linkhttps://assafbk.github.io/website/ calendar_today08-08-2023 05:55:05

59 Tweet

94 Takipçi

126 Takip Edilen

Assaf Ben Kish

@abk_tau

4 months ago

New work! 🚨 Recurrent LLMs like Mamba and RWKV can efficiently process millions of tokens, yet still underperform on real-world long-context tasks. What's holding them back? 🤔 And how can a lightweight fix boost their performance by 35% on LongBench? 👇🏼🧵 Github:

thumb_up_off_alt89

chat_bubble_outline3

repeat23

shareShare