PicoCreator - AI Model Builder 🌉 (@picocreator) 's Twitter Profile
PicoCreator - AI Model Builder 🌉

@picocreator

Serverless 🦙 @ featherless.ai Build Attention-Killers AI (RWKV) from scratch @ wiki.rwkv.com Also built uilicious & GPU.js (gpu.rocks)

ID: 884406703

linkhttps://featherless.ai calendar_today16-10-2012 13:02:54

3,3K Tweet

4,4K Followers

226 Following

PicoCreator - AI Model Builder 🌉 (@picocreator) 's Twitter Profile Photo
PicoCreator - AI Model Builder 🌉
@picocreator

4 months ago

❗️Attention is NOT all you need ❗️ Using only 8 GPU's (not a cluster), we trained a Qwerky-72B (and 32B), without any transformer attention With evals far surpassing GPT 3.5 turbo, and closing in on 4o-mini. All with 100x++ lower inference cost, via RWKV linear scaling

❗️Attention is NOT all you need ❗️ Using only 8 GPU's (not a cluster), we trained a Qwerky-72B (and 32B), without any transformer attention With evals far surpassing GPT 3.5 turbo, and closing in on 4o-mini. All with 100x++ lower inference cost, via RWKV linear scaling