jack morris (@jxmnop) 's Twitter Profile
jack morris

@jxmnop

getting my phd in language models @cornell_tech 🚠 // former researcher @meta // academic optimist // master of the semicolon

ID: 783098774130401280

linkhttp://jxmo.io calendar_today04-10-2016 00:17:51

4,4K Tweet

31,31K Takipçi

793 Takip Edilen

jack morris (@jxmnop) 's Twitter Profile Photo
jack morris
@jxmnop

4 months ago

pretty mind-blowing fact I just learned about transformer language models: the positional embeddings don't really do anything. you can just get rid of them and the model still works just as well sounds impossible, doesn't it? turns out standard LLMs aren't actually

pretty mind-blowing fact I just learned about transformer language models: the positional embeddings don't really do anything. you can just get rid of them and the model still works just as well sounds impossible, doesn't it? turns out standard LLMs aren't actually