Interesting holiday reading on the Verse PL :) simon.peytonjones.org/assets/pdfs/ha… simon.peytonjones.org/assets/pdfs/ve…

Very sad that Cashew passed away suddenly on Christmas Eve. She was 18 years old. We will miss her very much!

Some advice from physics laureate Frank Wilczek to kick off 2022.

Adding custom ops to JAX for GPUs jax.readthedocs.io/en/latest/Cust…

Our work "𝗗𝘆𝗻𝗮𝗺𝗶𝗰 𝗥𝗮𝗰𝗲 𝗗𝗲𝘁𝗲𝗰𝘁𝗶𝗼𝗻 𝘄𝗶𝘁𝗵 𝗢(𝟭) 𝗦𝗮𝗺𝗽𝗹𝗲𝘀" appears at POPL 2025 this year. Its a cute result demonstrating (yet again) how algorithmic techniques can be used to design simple effective solutions to traditionally hard systems-y problems.

Excited to announce that our paper "Graphene: An IR for Optimized Tensor Computations on GPUs" has been accepted at ASPLOS '23! Joint work with Bastian Hagedorn , Bin Fan, Hanfeng Chen, Cris Ceka and Michael Garland #asplos23

Pleased to share that our 2022 ISCA Paper 'Mixed-Proxy Extensions for the NVIDIA PTX Memory Consistency Model' has been awarded an Honourable Mention for IEEE Micro Top Picks 2023! dl.acm.org/doi/abs/10.114…

CUTLASS 3.0 has just been released, offering optimal performance on NVIDIA’s H100 and a new approach to template metaprogramming in CUDA C++. github.com/NVIDIA/cutlass

GPU viewed as a nd-tensor of threads

Our ASPLOS paper Graphene: An IR for Optimized Tensor Computations on GPUs is now available as open access here: dl.acm.org/doi/abs/10.114…

Presenting this at ASPLOS on Monday. :)

Looking to hire Compiler Researchers and Engineers Research Scientist - AI Compiler : nvidia.wd5.myworkdayjobs.com/NVIDIAExternal… Senior Software Engineer – AI Compilers for LLM: nvidia.wd5.myworkdayjobs.com/NVIDIAExternal…

Announcing FlashAttention-2! We released FlashAttention a year ago, making attn 2-4 faster and is now widely used in most LLM libraries. Recently I’ve been working on the next version: 2x faster than v1, 5-9x vs standard attn, reaching 225 TFLOPs/s training speed on A100. 1/

Compiler research intern positions at NVIDIA (Seattle) for ML. Functional programming, polyhedral compilation, program synthesis, compiler optimization. Send resume to [email protected]

[HOPE'23] Free Variable as Effect, in Practice youtu.be/Ce0NAYaO7tw by Oleg Kiselyov

The Evolution of Effects youtu.be/m821Vz8N_bo?si…

New blog post on "Lean Scala". odersky.github.io/blog/2024-04-1…