Naina Chaturvedi(@NainaChaturved8) 's Twitter Profileg
Naina Chaturvedi

@NainaChaturved8

World Traveler, Sr.SDE,Researcher Cornell Uni,ACM,Competitive Programmer,Google's WTM,Goldman Sachs 10K Women,Coursera Instructor,IITB,Grace hopper,53 countries

ID:1229991659812704257

linkhttps://naina0412.medium.com/ calendar_today19-02-2020 04:50:46

10,2K Tweets

6,7K Followers

3,5K Following

Andrej Karpathy(@karpathy) 's Twitter Profile Photo

New (2h13m πŸ˜…) lecture: 'Let's build the GPT Tokenizer'

Tokenizers are a completely separate stage of the LLM pipeline: they have their own training set, training algorithm (Byte Pair Encoding), and after training implement two functions: encode() from strings to tokens, and…

New (2h13m πŸ˜…) lecture: 'Let's build the GPT Tokenizer' Tokenizers are a completely separate stage of the LLM pipeline: they have their own training set, training algorithm (Byte Pair Encoding), and after training implement two functions: encode() from strings to tokens, and…
account_circle