Let yourself occasionally get a little screwed in exchange for not having to live with your guard up all the time. It’s worth it.

CUDA Graphs helped scale NVIDIA's @commons_ml workloads to over 4000 GPUs. Now available from PyTorch v1.10 onwards.

Excited about #JAX ? NVIDIA is hiring - please reach out if you would like to join or know someone who would be a great fit! tinyurl.com/46ujx4bf

Really

1/7 We built a new model! It’s called Action Transformer (ACT-1) and we taught it to use a bunch of software tools. In this first video, the user simply types a high-level request and ACT-1 does the rest. Read on to see more examples ⬇️

Distilled #StableDiffusion2 > 20x speed up, convergence in 1-4 steps We already reduced time to gen 50 steps from 5.6s to 0.9s working with NVIDIA Paper drops shortly, will link, model soon Will be presented @NeurIPS by Chenlin Meng & Robin Rombach Interesting eh 🙃

Excited to announce that our paper "Graphene: An IR for Optimized Tensor Computations on GPUs" has been accepted at ASPLOS '23! Joint work with Bastian Hagedorn , Bin Fan, Hanfeng Chen, Cris Ceka and Michael Garland #asplos23

Using CUTLASS int4b GEMM to train Transformer lnkd.in/gKFucNii

It has become commonplace to speak of the “existential risk” of AI. Recently even top AI scientists have begun to talk this way. I, for one, find it an unhelpful. So, without controversy, we can note: 1. AI scientists disagree about whether or not “existential risk of AI” is a

CUTLASS passed 3K Github stars last week. The recent releases of xFormers and FlashAttention FMHA are the last push to help us achieve this milestone. If you haven't done so, please leave us a star. github.com/NVIDIA/cutlass…

Compiler research intern positions at NVIDIA (Seattle) for ML. Functional programming, polyhedral compilation, program synthesis, compiler optimization. Send resume to [email protected]

We just got 1024 A100s up and running at Together AI!! We are offering short-term dedicated access to AI startups anywhere from 16-128 GPUs. Clusters come pre-configured with distributed training software. Available immediately (while supplies last) 🚀🚀🚀

If you are a CUTLASS fan , join our Discord - discord.com/invite/VbXssNTq NVIDIA AI Haicheng Wu

CUTLASS passed 20M downloads. The first 10M took 5 yrs. The 2nd took 5 months. BTW, we have an official CUTLASS discord channel now: discord.com/invite/Nh9hQhY7

Dear CUTLASS developers, we are looking for both full time and interns. If you already use CUTLASS to build something non trivial, it is easy for you to stand out from the crowd. nvidia.wd5.myworkdayjobs.com/en-US/NVIDIAEx…

JAX is great for GPUs too! With a strong team, and excellent execution, you can train a world-class LLM in ~two months!

.Lamini takes advantage of NVIDIA accelerated computing platform including #CUDA cores, Tensor Cores, CUDA-X Libraries and Nsight Tools to optimize their Memory Tuning algorithm. Read their blog 👇 nvda.ws/3WlGSjC

Really looking forward to collaborating and partnering with NVIDIA to further optimize our stack on NVIDIA hardware :)