🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

New Paper! 📣 RL^V: a unified RL & generative verifier —boosts MATH accuracy by 20% and improves both sequential and parallel test-time scaling ☑️ improves out-of-domain and easy-to-hard generalization ☑️ allows dynamic allocation of compute for harder problems How? 👇🏻

Idea: Merging generative verification and solution generarion during RL training of LLM reasoners. Why? This allows you to scale inference compute both sequentially (long CoT) and parallel (Best of N, weighted maj voting). Next? Generation and verification to be trained end

Releasing INTELLECT-2: We’re open-sourcing the first 32B parameter model trained via globally distributed reinforcement learning: • Detailed Technical Report • INTELLECT-2 model checkpoint primeintellect.ai/blog/intellect…

I suspect this paper's result have been oversold somewhat. As far as I can tell, nothing in the paper excludes the possibility that a quite large % of of the "learning" here is just "learns to be put answers in \\boxed{...}" tags.

Someone passed this wisdom to me today. Deep learning techniques working vs not working is two devils - your prior about the technique - your attention to details about implementation of the technique Need both to make it work.

All you often need is just one lucky break. For me, it was Geoffrey Hinton who took a bet on me about 7 years ago. He said something along the following lines that stuck with me: “You have tried a bunch of interesting research directions , and all of them failed — that’s what

An undergrad student broke a 40-year-old belief in computer science. Since 1985, it was believed that hash tables, when nearly full, must check many spots to find or add data. Andrew Krapivin discovered a new way to organize data inside a hash table that avoids this slowdown.

The reason recent RLVR papers show mostly formatting and not learning new skills is just because no one has scaled up enough. If RL compute is <.1% of overall compute, ofc not much changes. I bet o3 is closer to 5% of total compute. 10-25% i bet the models feel different again.

Mind the GAP! we've had a few works proposing techniques for enabling scaling in deep rl, such as MoEs, tokenization, & sparse training. Ghada Sokar and i looked further & found a bit more clarity into *what* enables scaling, leading us to simpler solutions (see GAP in figure)! 1/

For Deepseek-R1, I am trying to estimate how much training compute was used for RL vs pre-training. My current estimate is 12-18% of pre-training compute was used for RL training! Is this estimate off? Pre-training: 2788K H800 hours for Deepseek-V3 Base. BS = 6144 with 4K seq

100% agree! In our recent work, we show RLVR can even work with random rewards on Qwen2.5-Math. However, all these surprising phenomena are more of an artifact of certain models--not generalizable to models with different prior, also unlikely large scale🤔 x.com/StellaLisy/sta…

Random labels helping makes sense when P(correct|random)>0 since the x-change rate of accuracy is much higher for true positive rather than false negative/false positive examples. Wrong labels working makes little sense, and is a result of undertraining + trajectories being

I believe that we need a deeper understanding of the relationship between pre-training and RL scaling. How to perform pre-training better, making language models more suitable and smooth for RL scaling? That is to say, Pre-training for RL. If you are interested in it, welcome to

So many works talking about entropy, but what is the **mechanism** of entropy in RL for LLMs? 🤔 Our work gives a principled understanding, as well as two tricks that get entropy **controlled** 🧵

a great video by Jia-Bin Huang explaining kl divergence and its computation