I always find the weirdest fruits to try lol

Does any one else use coil whine for confirmation that something is actually happening?? I can’t tell if my gpus are talking or screaming in pain

layernorm has been updated to be parameterless, so we can do it without a weight or bias now! On to RMSNorm

RMSNorm is up! Just rope to go!

Also, Liger Kernels, y'all are incredible for open sourcing so many of these triton kernels! It has been an incredible learning tool! github.com/linkedin/Liger…

writing a distributed dataloader that can also save and resume training sucked... the more I build mytorch the more I appreciate the awesome people who made PyTorch!!

Have a few test runs going on a 560M Param model using only MyTorch! Having some stability issues im sorting through but this lo key might actually work!

training LLMs in fp16 really sucks... i wish CUPY had BF16 Support!

Policy Gradients are an extremely important idea that allows us to avoid doing a two step approach of first estimating Q values and then deriving the Policy from there. The main trick is how can we describe the derivative of our policy! Today we do that derivation and then a

EnCodec reimplementation video coming soon!