NVIDIA Grace whitepaper is online! More details of our new Arm based CPU. Glad to have had the opportunity to contribute along many other colleagues. Bioinformatic workloads will large benefit from Grace unique features and NV software stack support. bit.ly/nv-grace-wp

Epic with #genomics! Thanks Lotfi Slim and Rabab Alomairy, Ph.D. for this cool collaboration. Linear algebra comes again to rescue and addresses multiple precisions during genome-wide association studies. #GWAS Do not oversolve! Ask questions live on Mar 23 at 2pm CET. #HPC #GTC23

At today's #GTC23, PacBio's COO Mark van Oene will present performance gains by porting rate-limiting CPU steps of HiFi generation to NVIDIA GPUs and how we productized Google Health's DeepConsensus, including it in each and every Revio instrument.

1/ We are happy to announce the open-source release of the inference code and weights of our four genomics #LLM, the nucleotide transformers, ranging from 500M to 2.5B parameters, trained in collaboration with NVIDIA and @TU_Munchen 🧬 bit.ly/3YRSUz1

Great work from Hassan Sirelkhatim and Christian Dallago as always!

A worthy competitor to the TPU pods has (finally) arrived: developer.nvidia.com/blog/announcin…

H100 GPUs are very fast! For those unfamiliar with GPU matrix multiplies, the jaggies in the graph relate to packing occupancy, and are not noise. You can’t just divide theoretical teraflops by your problem size and get accurate times.

We’re proud to have helped set a new standard for Generative AI by building a cluster with 3,584 @NVIDIA H100 GPUs in the debut MLPerf benchmark. Our system completed the massive GPT-3-based training benchmark in under eleven minutes. 🚀 Read more: blogs.nvidia.com/blog/2023/06/2…

A must-read blogpost about sequence alignment and the differentiable CTC loss distill.pub/2017/ctc/

Completely blown away by template programming in C++ being Turing complete.

I've done a deep dive into distributed training and efficient fine-tuning of LLMs. I get into the messy internals of DeepSpeed ZeRO and FSDP, summarize practical guidelines and highlight gotchas with multi-GPU training. sumanthrh.com/post/distribut… Do read, should be fun!

Two new chapters from my open-access textbook in human population genetics are now online: - Population structure: I. Ancestry estimation - Population structure: II. More about admixture web.stanford.edu/group/pritchar…

We gave our first in depth publicly available talk on CUTLASS 3.x and it’s up on YouTube now!

pytorch.org/blog/int4-deco… I highly recommend this study to build intuitions for GPU programming. One of the difficulties with GPU programming is that it's hard to test a hypothesis about a certain optimization. Say we expect an optimization to improve the final speed, but we

📣 When #exascale meets #bioinformatics #GWAS to capture nonlinear genetic #epistasis using #NVIDIA #MxP #GPU: - 1.8 EFlop/s on #Alps CSCS Lugano - 305K patients of UK #BioBank - 13M synth cohort, ~63% of the world's countries 🌍 - 10000X vs SOTA #REGENIE #GordonBell #SC24 #HPC

Come learn today at 2pm ET why we use NVIDIA AI CUTLASS for high-throughput inference kernels in vLLM! If you are interested in peak quantized GEMM performance on GPUs, this is the talk to attend and ask questions

50% increase in transformer throughput via the use of NVIDIA cutlass to implement fused quantised layers #nanoporeconf

BREAKING NEWS The Royal Swedish Academy of Sciences has decided to award the 2024 #NobelPrize in Literature to the Attention Is All You Need authors. Their work has made thousands cry, laugh, or rich and made GPUs go brrr

🔥🚨 CUTLASS Blackwell is here 🚨🔥 3.8 release is loaded with support for new features of Blackwell, even an attention kernel 👀 Go check it out here: github.com/nvidia/cutlass Can't wait to see what y'all end up cooking with this over the next few moths and years 💚