Maybe to one's surprise, taking KL estimates as `kl_loss` to minimize does *not* enforce the KL. This implementation, however, is quite common in open source RL repos and recent research papers. In short: grad of an unbiased KL estimate is not an unbiased estimate of KL grad.

😭😂

Gonna make my own Sapphire fr! 😭

so reward hacking gonna brrrrr. will brown will be joining us for the next pod ⚡ you're not ready for this. more soon!

Holy shit! 0 college attendance in last sem.

when a new shape comes :)

Too early to be sure but I'm trying to use optimal control ala RB Modulation but replacing consistency loss with reward signal as a kind of test time scaling technique. Baseline vs With reward model.

This is sick!

every gpu kernel is a logical subdivision of divide and conquer

New paper on the generalization of Flow Matching arxiv.org/abs/2506.03719 🤯 Why does flow matching generalize? Did you know that the flow matching target you're trying to learn **can only generate training points**? with Quentin Bertrand, Anne Gagneux & Rémi Emonet 👇👇👇

Using MXFP8 on 5090 to get 2x speedup on Flux!

why is it only 3hrs 😭

subho ghosh true. was just noting down all the topics. piyushmaharanacats.blogspot.com/2025/06/rabbit…

Hmm, Still using the LAION Aesthetic reward on Flux. You can tell the difference, but it doesn't feel quite there yet. Will play around with couple of other reward models and their combinations. This is relative reward by the way. Much more stable vs absolute reward.

midjourney cooked!!

the FP8 values in your model after 50 layers of quantize/dequantize operations

gm guys!

all hail Aryan 🫡

One of the best ways to reduce LLM latency is by fusing all computation and communication into a single GPU megakernel. But writing megakernels by hand is extremely hard. 🚀Introducing Mirage Persistent Kernel (MPK), a compiler that automatically transforms LLMs into optimized