AI Compiler research intern positions at NVIDIA (Seattle). Functional programming, polyhedral compilation, program synthesis, compiler optimization. Send resume to [email protected]

Preprint arxiv.org/pdf/2412.13398

We are excite to announce FlashInfer v0.2! Core contributions of this release include: - Block/Vector Sparse (Paged) Attention on FlashAttention-3 - JIT compilation for customized attention variants - Fused Multi-head Latent Attention (MLA) decoding kernel - Lots of bugfix and

Amazing to see Flashinfer’s traction in the short 8mo since it was first introduced. Try out the latest release.

Scaling Deep Learning Training with MPMD Pipeline Parallelism. Joint work with Anxhelo Xhebraj Sean Lee Hanfeng Chen arxiv.org/abs/2412.14374

Check out Vinod Grover and team's work on scaling JAX based DL Training

Latest version of flashInfer paper with some cool ideas!

🔥🚨 CUTLASS Blackwell is here 🚨🔥 3.8 release is loaded with support for new features of Blackwell, even an attention kernel 👀 Go check it out here: github.com/nvidia/cutlass Can't wait to see what y'all end up cooking with this over the next few moths and years 💚

Pipeline Parallelism in JAX!

If you would like to share your work on array programming please consider submitting your paper to ARRAY '25 (co-located with PLDI ). pldi25.sigplan.org/home/ARRAY-202…

Accepted for publication in #MLSys25 conference!

Check out the intra-kernel profiler in flashinfer to visualize the timeline of each SM/warpgroup in the lifecycle of a CUDA persistent kernel: github.com/flashinfer-ai/… You can clearly understand how tensor/cuda cores overlapping, variable length load-balancing and fusion works.

#NewProfilePic

Happy to share our latest work at ASPLOS 2025! LLMs are dynamic, both in sequence and batches. Relax brings an ML compiler IR that globally tracks symbolic shapes across functions on multiple levels. Bring efficient and flexible LLM AOT compilation arxiv.org/abs/2311.02103.

Congratulations to Swarat Chaudhuri (PhD, CIS@Penn 2007) for this wonderful honor from Guggenheim Foundation gf.org/stories/announ…

This is an SM90 to SM100 porting guide deep research and i made that is mostly accurate. sharing in case others might find it useful x.com/_xjdr/status/1…

MLSys 2025 is coming up! Want to meet the developers behind FlashInfer, XGrammar, and SGLang LMSYS Org in person? Join us for the Happy Hour on May 12—we’d love to see you there! lu.ma/dl99yjoe

#MLSys2025 make sure to attend 10:30am keynote Ion Stoica An AI stack: from scaling AI workloads to evaluating LLMs. Checkout full schedule at mlsys.org/virtual/2025/c…

Here's a paper describing quantum computing using standard programming constructs, w/o linear algebra! Goal: demystify quantum computing + serve as a formal foundation for reasoning about quantum programs. paper paper eprint.iacr.org/2025/1091.pdf code github.com/qqq-wisc/qwla