Thank you for highlighting our work!

Don’t like to just put the paper together too, so trying to deliver some insight🤣

Long texts choke transformers in LLMs, and this study proves that weaving a few full attention layers into mostly linear ones keeps memories sharp without the huge cache. The team trained 72 models up to 1.3B parameters, testing 6 linear designs across several mixing ratios.

Most current language models think out loud, stuffing every thought into words. A typical token set holds about 40000 choices, which equals roughly 15 bits of data, just under 2 bytes. When a language model must pour every reasoning step through these tiny packets, complex

Nice work by Jiashuo’s team!

Is text-only information enough for LLM/VLM Web Agents? 🤔 Clearly not. 🙅‍♂️ The modern web is a rich tapestry of text, images 🖼️, and videos 🎥. To truly assist us, agents need to understand it all. That's why we built MM-BrowseComp. 🌐 We're introducing MM-BrowseComp 🚀, a new