turbopuffer (@turbopuffer) 's Twitter Profile
turbopuffer

@turbopuffer

search every byte 🔍 {vector, full-text} search engine built from first principles on object storage. 10x cheaper & scales to 100B. power Notion, Cursor, Linear

ID: 1709608373275017219

linkhttps://turbopuffer.com calendar_today04-10-2023 16:36:47

145 Tweet

3,3K Takipçi

2 Takip Edilen

turbopuffer (@turbopuffer) 's Twitter Profile Photo
turbopuffer
@turbopuffer

3 months ago

30x daily maintenance job speedup from one simple hack: 41h → 82m 🧹 Bottleneck was paging through S3/GCS prefixes (1:1 w/namespaces). Added a read-through cache of every Nth prefix (N=page_size/2) so now we can list with arbitrary concurrency.

30x daily maintenance job speedup from one simple hack: 41h → 82m 🧹 Bottleneck was paging through S3/GCS prefixes (1:1 w/namespaces). Added a read-through cache of every Nth prefix (N=page_size/2) so now we can list with arbitrary concurrency.