welcome to pupo

today

With the awesome Ramón Calvo, Daniele Paliotta, Matteo Pagliardini, and Martin Jaggi. Faculty of Science | UNIGE EPFL Computer and Communication Sciences TL;DR: you can shuffle the middle layers of a transformer without retraining it. We take advantage of that to compute layers in parallel. arxiv.org/abs/2502.02790

Check out our new work!

Thinking Slow, Fast: Scaling Inference Compute with Distilled Reasoners Distilling Llama-1B and -3B models with only 8 billion tokens into subquadratic models like Mamba to achieve better and faster scaling of inference-time compute with minimal performance loss.

Mamba's higher inference throughput means you can sample more CoTs for reasoning and get a better model for the same budget. I've been thinking about the ideal architecture for test-time compute, more on this soon!

*Leveraging the true depth of LLMs* by Ramón Calvo Daniele Paliotta Matteo Pagliardini François Fleuret Based on the observation that layer shuffling has little impact on the middle layers of LLMs, they propose to parallelize their execution in pairs. arxiv.org/abs/2502.02790

sliding widow idk

M1: Towards Scalable Test-Time Compute with Mamba Reasoning Models - >3x faster inference over Transformer - Outperforms distilled R1 under a fixed generation time budget

M1 is a hybrid Mamba-based reasoning model designed to scale test-time generation more efficiently than Transformers. - built via distillation from LLaMA3.2-3B and enhanced with SFT + RL (GRPO) - trained on <50B tokens vs. DeepSeek-R1’s 1T+ MATH tokens - 3× faster inference vs.

M1 Towards Scalable Test-Time Compute with Mamba Reasoning Models

⏳Transformer-based LLM reasoning is effective but slow. Distilled Mamba reasoners have ~4x throughput vs teachers, enabling better coverage and accuracy at many time budgets! 📍Reasoning and Planning for LLMs Workshop Paper: arxiv.org/abs/2502.20339 x.com/DanielePaliott… 3/3

Gemini knows, gamble responsibly #RolandGarros