Andrej Karpathy(@karpathy) 's Twitter Profileg
Andrej Karpathy

@karpathy

🧑‍🍳. Previously Director of AI @ Tesla, founding team @ OpenAI, CS231n/PhD @ Stanford. I like to train large deep neural nets 🧠🤖💥

ID:33836629

linkhttps://karpathy.ai calendar_today21-04-2009 06:49:15

8,5K Tweets

921,9K Followers

889 Following

Andrej Karpathy(@karpathy) 's Twitter Profile Photo

Very interesting idea. I tried a custom version of it with a simple prompt and it worked really well out of the box. Basically, GPT is surprisingly good at correcting minor typos, so you can write really really fast, ignore mistakes and keep going, and it comes out just fine.

account_circle
Charles 🎉 Frye(@charles_irl) 's Twitter Profile Photo

PagedAttention, Virtual Context, Speculative Decoding, Register Tokens: the last year has seen many ideas from systems programming applied to LLMs.

Not many folks live in that intersection, so I wrote an explainer post to make them a bit more accessible!

charlesfrye.github.io/programming/20…

PagedAttention, Virtual Context, Speculative Decoding, Register Tokens: the last year has seen many ideas from systems programming applied to LLMs. Not many folks live in that intersection, so I wrote an explainer post to make them a bit more accessible! charlesfrye.github.io/programming/20…
account_circle
Andrej Karpathy(@karpathy) 's Twitter Profile Photo

LLM OS. Bear with me I'm still cooking.

Specs:
- LLM: OpenAI GPT-4 Turbo 256 core (batch size) processor @ 20Hz (tok/s)
- RAM: 128Ktok
- Filesystem: Ada002

LLM OS. Bear with me I'm still cooking. Specs: - LLM: OpenAI GPT-4 Turbo 256 core (batch size) processor @ 20Hz (tok/s) - RAM: 128Ktok - Filesystem: Ada002
account_circle
Andrej Karpathy(@karpathy) 's Twitter Profile Photo

Original copilot was ~few line tab autocomplete.
GPT-like chatbots now routinely do larger chunks.
Then get PRs given Issues.
Then write the Issues.
Human input and oversight gradually ascends in abstraction and contributes less, until it is ~pass-through.
githubnext.com/projects/copil…

account_circle
Andrej Karpathy(@karpathy) 's Twitter Profile Photo

Seek to ~1hr mark.
With the newly announced GPTs, I think we’re seeing a new (still a bit primordial) layer of abstraction in computing. There will be a lot more developers, and a lot more GPTs. GPTs that can read, write, hear, speak, see, paint, think, use existing computing as…

account_circle
Andrej Karpathy(@karpathy) 's Twitter Profile Photo

ChatGPT 'Advanced Data Analysis' (which doesn't really have anything to do with data specifically) is an awesome tool for creating diagrams. I could probably code these diagrams myself, but it's soo much better to just sit back, and iterate in English.

In this example, I was…

ChatGPT 'Advanced Data Analysis' (which doesn't really have anything to do with data specifically) is an awesome tool for creating diagrams. I could probably code these diagrams myself, but it's soo much better to just sit back, and iterate in English. In this example, I was…
account_circle
Andrej Karpathy(@karpathy) 's Twitter Profile Photo

It is a highly amusing (personal) historical quirk that I was very excited about language models in 2015 (and this blog post on them made rounds), but when we started OpenAI few months later the thought hasn't crossed my mind to work on them. I was very interested in RL. lol sigh

account_circle
Andrej Karpathy(@karpathy) 's Twitter Profile Photo

State of AI Report: very nice snapshot of the AI ecosystem across research, industry and (geo)politics (as usual each year :)). stateof.ai

account_circle
Andrej Karpathy(@karpathy) 's Twitter Profile Photo

'The Tyranny of the Marginal User'
Why consumer software gets worse, not better, over time. Great post from ivan, hard to not see it everywhere.

'Here’s what I’ve been able to piece together about the marginal user. Let’s call him Marl.'

nothinghuman.substack.com/p/the-tyranny-…

account_circle
Andrej Karpathy(@karpathy) 's Twitter Profile Photo

An OS that boots to a baby Llama 2
github.com/trholding/llam…
Standalone, Binary Portable, Bootable

I expected that my 'Llama 2 inference code in a single .c file' would go places, but this really stretches the imagination :) And why not, do we really need all this stuff?

An OS that boots to a baby Llama 2 github.com/trholding/llam… Standalone, Binary Portable, Bootable I expected that my 'Llama 2 inference code in a single .c file' would go places, but this really stretches the imagination :) And why not, do we really need all this stuff?
account_circle