Assaf's analysis of recurrent LLMs such as Mamba and RWKV is important. While these models are designed to be efficient for long-context tasks, their effectiveness remains limited due to memory overflows - even at large scale. More details about memory overflows in the paper📜🧵

How do information flow patterns in Mamba compare to those in Transformers?🚨 Our new #ACL2025 paper pits Mamba-1 and Mamba-2 against other Transformer-based models and uncovers both universal and architecture-specific information flow patterns. arxiv.org/abs/2505.24244 🧵

Revisiting LRP: Positional Attribution as the Missing Ingredient for Transformer Explainability 𝘚𝘪𝘮𝘱𝘭𝘦 𝘧𝘪𝘹𝘦𝘴 > 𝖯𝗋𝗈𝖻𝗅𝖾𝗆 All LRP‑based XAI tools ignore PE, so relevance gets lost 𝖥𝗂𝗑 Model each input as a (token, position) pair and add PE‑aware LRP rules

Reasoning depth of LLMs can now match task size, not a fixed budget. Reasoning models waste compute and often slip because their hidden chain of thought keeps running after the answer is clear. The authors learn an internal progress meter and nudge it so the model stops as

LLMs Don’t Think Like Developers - Until Now. Together with shahar katz and liorwolf We made LLMs execute their code while generating it, just like a human developer. Meet EG-CFG: A new inference-time method that injects real-time execution feedback into the generation loop.

Exciting news from #ICML2025 & #ICCV2025 🥳 - 🥇 VideoJAM accepted as *oral* at #ICML2025 (top 1%) - Two talks at #ICCV2025 ☝️interpretability in the generative era ✌️video customization - Organizing two #ICCV2025 workshops ☝️structural priors for vision ✌️long video gen 🧵👇