profile-img
Andrej Karpathy

@karpathy

🧑‍🍳. Previously Director of AI @ Tesla, founding team @ OpenAI, CS231n/PhD @ Stanford. I like to train large deep neural nets 🧠🤖💥

calendar_today21-04-2009 06:49:15

8,7K Tweets

979,4K Followers

905 Following

Andrej Karpathy(@karpathy) 's Twitter Profile Photo
Andrej Karpathy
@karpathy

2 weeks ago

Okay I did a first quick pass of naive CUDA kernels for the forward pass of GPT-2 and pushed everything to one file in llm.c, Still only ~1000 lines of code:
github.com/karpathy/llm.c…

Current per iteration timings on my Lambda box <3 A100 40GB PCIe, B=4, T=1024:
- llm.c: 111ms
-

Okay I did a first quick pass of naive CUDA kernels for the forward pass of GPT-2 and pushed everything to one file in llm.c, Still only ~1000 lines of code: github.com/karpathy/llm.c… Current per iteration timings on my Lambda box <3 A100 40GB PCIe, B=4, T=1024: - llm.c: 111ms -
account_circle