We've kicked off the NeurIPS Meta Hacker Cup AI track! I mostly use LLMs to code and they've been great at removing the drudgery of boiler-platy code but for harder problems they are meh Both the big-boi llamas and GPT-4 fall flat on their face!

Check out the first AI track of the Meta Hacker Cup ! 5 rounds, some of the hardest problems in code generation, same problems released to human and AI competitors this year. Large Discord group with competitors, just getting started codeforces.com/blog/entry/131…

I'm beyond stoked to be hosting the first IRL hackathon for the CUDA MODE community with Accel NVIDIA and PyTorch ! Get ready to hack across a range of AI performance -- inference optimization, custom kernels, low bit dtypes, sparsification, ...

We recently enabled quantization-aware training in torchtune using torchao. QAT helps recover any quality degradation resulting from quantizing the model - especially useful when you’re resource constrained at inference time. 👇🏽awesome work by Andrew Or! pytorch.org/blog/quantizat…

Registration is open for the #NeurIPS2024 HackerCup, the world's first AI programming competition by Microsoft Research, Meta, and PyTorch. Test your AI on the programming problems that challenge the best human minds and see how your AI ranks. Learn more. msft.it/6013lm3f3

Excited for our release of LeanRL, showcasing the simplicity and performance of torch compile across a breath of RL Models. Big shout out to Costa Huang for the inspiration and support

From the beginning, Andreas Köpf and I wanted a community that was cozy, inclusive and fun where you could make friends and learn how to make your code faster Anyone who shares our ideals is welcome. We’re announcing we’re changing the name of our community to “GPU MODE”.

A huge thanks to Casey Aylward and the Accel team for running the highest quality sys ml hackathon I’ve ever see . It was great to meet so many GPU MODE friends IRL, and hack on torchao. And Congrats to the winners for fast flexible attention!

PyTorch has historically been super slow for a lot of "small" RL workloads, but the reason is kinda dumb. RL often uses tiny neural networks, and end up ridiculously CPU overhead bound. Luckily, with cudagraphs and torch.compile, we often see >5x speedups! I remember the first

We ran a lot of evals recently using Mistral Large 2 for the hacker cup challenges, Weave’s native integration with the Mistral SDk was super useful to understand token usage and latency, as well as measure accuracy

Congrats on the launch of 🌽 KernelBench! We're excited to partner with 🍿 Popcorn to lower the barrier to GPU programming!

Stoked for the Hacker AI Cup Workshop to start at NeurIPS Conference ! Speakers Weiwei Yang Gabriel Synnaeve Neal Wu Simran Arora Misha Laskin Thomas Capella Mark Saroufim ! Come join, room 210

David Harmeyer on Facebook’s history of Hacker Cup. How to add N flights to N airports to minimize travel distance.

Great talk on the importance of RLEF and CoT for coding generation performance

Excited to share the GPU MODE kernel authoring competition! We’re stoked to see the creative solutions people come up with to push bounds of performance. And, all for a good cause — towards a public dataset of performant kernels 😁

🚀🚀🚀 come work with PyTorch on GPU kernel optimization, distributed training/inference/post-training, compiler design, ABI stability

Would you rather have better documentation for PyTorch prototype features but more BC breakages or secret APIs until we have more confidence?