🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Happy Throughput Thursday! We’re excited to release Tokasaurus: an LLM inference engine designed from the ground up for high-throughput workloads with large and small models. (Joint work with Ayush Chakravarthy, Ryan Ehrlich, Sabri Eyuboglu, Bradley Brown, Joseph Shetaye,

📢 Can't be more excited about this scaling law study. It reveals two important points: (1) The current Test-Time Strategies are not scalable (bottlenecked by O(N^2) memory access) w.r.p. to the nature of hardware (FLOPS grows much faster than memory bandwidth) (2) While

😜

You gotta love those scaling laws! Very insightful work. Also one of the few applications of sparsity that seems to work really well. Would love to see more work like this!

Kinetics: Rethinking Test-Time Scaling Laws Dense strategies like BoN and Long-CoT hit an O(N²) KV bottleneck, TTS isn’t FLOP-bound 𝘢𝘯𝘥 𝘯𝘦𝘷𝘦𝘳 𝘳𝘦𝘢𝘭𝘭𝘺 𝘸𝘢𝘴 - block top-k sparse attention cuts per-token cost - enables longer generations and more parallel trials on

Really nice work based on inference scaling laws that account for memory accesses. Very insightful!

Real-time video generation is finally real — without sacrificing quality. Introducing Self-Forcing, a new paradigm for training autoregressive diffusion models. The key to high quality? Simulate the inference process during training by unrolling transformers with KV caching.

Ah finally someone studied this!!! Such a pain lol

Our view on test-time scaling has been to train models to discover algos that enable them to solve harder problems. Amrith Setlur & Matthew Yang's new work e3 shows how RL done with this view produces best <2B LLM on math that extrapolates beyond training budget. 🧵⬇️

Say hello to Multiverse — the Everything Everywhere All At Once of generative modeling. 💥 Lossless, adaptive, and gloriously parallel 🌀 Now open-sourced: multiverse4fm.github.io I was amazed how easily we could extract the intrinsic parallelism of even SOTA autoregressive

🚀 Super excited to share Multiverse! 🏃 It’s been a long journey exploring the space between model design and hardware efficiency. What excites me most is realizing that, beyond optimizing existing models, we can discover better model architectures by embracing system-level

The calculation of the scaling is, unfortunately, wrong. As we discussed in a recent paper, Kinetics, arxiv.org/abs/2506.05333, the bottleneck of inference time scaling is the KV memory access, rather than FLOPs! Unless your target scenario is Ollama for a single user. (That's

Hello MiniMax (official) exciting model but questionable claim on its better reasoning scaling than DeepSeek and Qwen. Nice try on reasoning longer to be SOTA but using flops to quantify the cost in Test-time scaling doesn’t work for hybrid model 🫣 chen zhuoming has

What a beast 🤯

Xinyu Yang will be presenting this amazing work at ASAP seminar tomorrow! Do not miss his talk

Rollouts are a major bottleneck in RL training for LLMs. Our new proposed RL training method, GRESO, lets RL focus on high-value prompts— large saving rollout time, and accelerating training. 🚀

📣 Folks who’re seriously building scalable RL algorithms or systems need to pay attention to this 👇

wow 🤩 check this out!!!

This is cool!!!