Excited to introduce VTool-R1! We’ve trained VLMs to “think visually” using RL, blending Python-based 🖼️visual edits with💡textual Chain-of-Thought reasoning. Our trained qwen2.5-VL-32B surpasses GPT-4o on ChartQA & TableVQA, and even the compact qwen2.5-VL-7B significantly

Attention was never enough. The hybrid LLM era is here—and it’s moving fast. From Mamba to Jamba to Bamba, we mapped every major model that’s challenged the Transformer default in the past 18 months. 🧵 A timeline of what’s changed and why it matters ↓ 🔗

Failing on 𝐥𝐚𝐫𝐠𝐞-𝐬𝐜𝐚𝐥𝐞 𝐑𝐋 with VeRL? ⚠️ Mixing inference backend (𝐯𝐋𝐋𝐌/𝐒𝐆𝐋𝐚𝐧𝐠) with training backends (𝐅𝐒𝐃𝐏/𝐌𝐞𝐠𝐚𝐭𝐫𝐨𝐧) 𝐬𝐞𝐜𝐫𝐞𝐭𝐥𝐲 𝐭𝐮𝐫𝐧𝐬 𝐲𝐨𝐮𝐫 𝐑𝐋 𝐢𝐧𝐭𝐨 𝐨𝐟𝐟-𝐩𝐨𝐥𝐢𝐜𝐲 — even if they share the same weights! 📉 Blog:

kalomaze Its phi5-21B-A3.6B with phi5-117B-A5.1B

Can VLMs learn to reason better by drawing on the brilliant thoughts of others. 🔥Our recent work on vision language model reasoning, through carefully designed multimodal memory and retrieval, has been accepted to Main Conference of #EMNLP2025. 💡Inspired by case-based

(1/n) Check out our new paper: "Fantastic Pretraining Optimizers and Where to Find Them"! >4000 models to find the fastest optimizer! 2× speedups over AdamW? Unlikely. Beware under-tuned baseline or limited scale! E.g. Muon: ~40% speedups <0.5B & only 10% at 1.2B (8× Chinchilla)!

github.com/huggingface/tr…

Today Thinking Machines Lab is launching our research blog, Connectionism. Our first blog post is “Defeating Nondeterminism in LLM Inference” We believe that science is better when shared. Connectionism will cover topics as varied as our research is: from kernel numerics to

Excited to see Gated DeltaNet being adopted in the Qwen series ! It has also previously demonstrated strong effectiveness in NVIDIA's Jet-Nemotron

SambaY is accepted to #NeurIPS2025 ! Looking forward to seeing everyone in San Diego!

a well written pedagogical blog post. Some questions out of my curiosity: 1. for stably training large models, why is normalizing weights better than normalizing activations? 2. how much does does regularizing weight matrix to Stiefel manifold limit its expressivity +

Introducing Tinker: a flexible API for fine-tuning language models. Write training loops in Python on your laptop; we'll run them on distributed GPUs. Private beta starts today. We can't wait to see what researchers and developers build with cutting-edge open models!

It is really amazing to see a 5-year-old project finally got wrapped up and still seems very relevant to today's agentic research topics such as multi-agent collaboration, environment simulators and instruction following!

Our latest post explores on-policy distillation, a training approach that unites the error-correcting relevance of RL with the reward density of SFT. When training it for math reasoning and as an internal chat assistant, we find that on-policy distillation can outperform other

Wow, it is also nine days earlier than Agarwal et al.'s on-policy distillation work.

NorMuon from Zichong Li et al. takes the crown as the leading NanoGPT speedrun optimizer! github.com/KellerJordan/m… NorMuon enhances Muon with a neuron normalization step after orthogonalization using second-order statistics. arxiv.org/abs/2510.05491

Kimi Linear Tech Report is dropped! 🚀 huggingface.co/moonshotai/Kim… Kimi Linear: A novel architecture that outperforms full attention with faster speeds and better performance—ready to serve as a drop-in replacement for full attention, featuring our open-sourced KDA kernels! Kimi

Many people are confused by Minimax’s recent return to full attention - especially since it was the first large-scale pivot toward hybrid linear attention - and by Kimi’s later adoption of hybrid linear variants (as well as earlier attempts by Qwen3-Next, or Qwen3.5). I actually

just noticed modded-nanogpt adopt 'NorMuon' as default (?). it looks like `AdaMuon`. i personally didnt buy this idea because i thought Muon is enough and dont want to introduce optim state for 2nd moment again like adam... hmm arxiv.org/abs/2510.05491 arxiv.org/abs/2507.11005…

Hi Jeff Dean, what’s the plan for releasing the code for this line of work? None of these papers so far seem to have released any code