everyone betting me to lose but gonna make r1-1.5b as cheap as xgboost

is there a r1 distilled but 0.5b, would beat xgboost price

u can just copy paste python code calling o1 structured outputs api and print response.choices[0].message.parsed and use that as ur qwen0.5bcoder prompt, which performs better than naturaln lang prompt on chat based instruct model. this is especially true for low entropy task

hyperfitting is very real and useful for structured data extraction

small models need hyperfit for reasoning, some paper will talk about it in a few months, just some observations in the trenches

curriculum learning will be popular again, all post training is better curriculum learning

wizard of oz start to fall, long put to hedge before too late

one simple reward based on offline eval metric, no fancy formatting, grpo just works on 0.5B for structured data extraction, key is to hyperfit on small high quality sample first

teach ur small lm like an infant and it will be smarter than adults

u dont need format reward for grpo as long as u have a reasonably finetunes 0.5b model to start with, u would use constrained generation anyway at inference time.

4o insge is pretty good at philosophy - RL interpretation of strong church turing thesis <> Formal causes