measure twice, cut once
think twice, talk once
review twice, merge once

awesome performance profiling talk from a solana core dev 🔥

seeing a ton of AI engineers pivoting into to crypto recently

my advice: learn rust - its everywhere in this industry

- theres only so much you can do with python
- start with rust smart contracts (svm)
- then move to rust L1s (solana/reth/aptos/sui/...)

in the next version of Sig 🤖

well support downloading snapshots with minimum download speed requirements and trusted peers

thread for how it works under-the-hood /N ↓

value is created through constant iteration

if your iteration cycle is slow - ngmi

especially in code and hiring
- how fast can you go from idea to merged on main
- how fast can you go from intro call to hired

the SnapshotHash structure is a gossip datatype which contains
- the largest full snapshot
- and a list of incremental snapshots

when downloading,
- we prioritize larger slots
- and only download snapshots whos hashes are the same as our trusted peers

github.com/Syndica/sig/bl…

then for each peer, we download with the following urls:
- full:
http://{ip}:{rpc-port}//snapshot-{slot}-{hash}.tar.zstd

- incremental:
http://{ip}:{rpc-port}//incremental-snapshot-{base_slot}-{slot}-{hash}.tar.zstd

if the download speed is slow, we try another peer

especially relevant link on building products through iteration - and how tech debt is an exponential slow down on value creation

x.com/1ultd/status/1…

if you find this stuff interesting

come join the sig team and help build a new solana validator from scratch in zig

jobs.ashbyhq.com/syndica/15ab4e…

lil perp👾 dont listen to em - u and the team are killin it 💪

for example, if the network is on slot 100, the full snapshot could contain all accounts at slot 75, and a matching incremental snapshot could contain all accounts that changed between slot 75 and slot 100

first, theres two types of snapshots: full snapshots and incremental snapshots

full snapshots include all the accounts on the network at some specific slot

incremental snapshots are smaller and only contain the accounts which changed from a full snapshot.

to download a snapshot, gossip is started up to find other nodes in the network and collect gossip data

we look for peers who:
- have a matching shred version (ie, the network version/hard-forks)
- have a valid rpc socket (ie, can download from)
- and have a SnapshotHash

coding in zig

and thats all - thanks for reading till the end :)

all the code can be found in src/accountsdb/download.zig

github.com/Syndica/sig/bl…

BW yeah we take the same trusted peer approach the rust client does

Richard Patel how do you think about this for fd? since you guys are building in the open

x19 think twice reply guy here

BW we wait 30seconds before evaluating the speed

we also sample 1k peers from the gossip table each iteration - the loop doesnt exit until a snapshot's downloaded

thats what we got rn but still evaluating different parameters