mike64_t (@mike64_t) 's Twitter Profile
mike64_t

@mike64_t

descending the gradient

ID: 1581044963114209286

calendar_today14-10-2022 22:11:31

2,2K Tweet

2,2K Takipçi

282 Takip Edilen

mike64_t (@mike64_t) 's Twitter Profile Photo
mike64_t
@mike64_t

19 days ago

A 256x256 matrix memory at 2048 tokens of sequence length can achieve 98.6% retrieval accuracy. With random embeddings, accuracy would saturate at 67% and with optimized embeddigs at 80%. Combined with a 4-layer LSTM decoder and a 1-layer store gate, the accuracy increases to

A 256x256 matrix memory at 2048 tokens of sequence length can achieve 98.6% retrieval accuracy. With random embeddings, accuracy would saturate at 67% and with optimized embeddigs at 80%. Combined with a 4-layer LSTM decoder and a 1-layer store gate, the accuracy increases to