Andrej Karpathy (@karpathy) 's Twitter Profile
Andrej Karpathy

@karpathy

Building @EurekaLabsAI. Previously Director of AI @ Tesla, founding team @ OpenAI, CS231n/PhD @ Stanford. I like to train large deep neural nets 🧠🤖💥

ID: 33836629

linkhttps://karpathy.ai calendar_today21-04-2009 06:49:15

9,9K Tweet

1,2M Followers

972 Following

Andrej Karpathy (@karpathy) 's Twitter Profile Photo
Andrej Karpathy
@karpathy

3 years ago

having fun optimizing minGPT today - base: 495ms - zero_grad(set_to_none=True): 492 - torch.jit.script gelu: 463 - OMP_PROC_BIND=CLOSE: 453 - torch.backends.cuda.matmul.allow_tf32: 143 - torch.autocast(torch.bfloat16): 121 - FlashAttention: 102 now: more fused kernels more better