Alexander Long (@_alexanderlong) 's Twitter Profile
Alexander Long

@_alexanderlong

Founder @PluralisHQ | prev. Applied Scientist at Amazon | ML PhD |
Protocol Learning: Multi-participant, low-bandwidth model parallel.

ID: 1684127558563151873

linkhttp://Pluralis.ai calendar_today26-07-2023 09:05:01

289 Tweet

1,1K Takipçi

715 Takip Edilen

Alexander Long (@_alexanderlong) 's Twitter Profile Photo
Alexander Long
@_alexanderlong

4 months ago

My view has always been if you give every node a copy of the full model weights, its not clear how you can introduce rational economics to decentralized training (since anyone can take the model). If you solve model parallel, no-one sees a full weight set.