🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Messing around with protocol buffers and the new tools directive in Go 1.24. It works pretty nicely with Buf

What if I told you that a 1 GiB/s Kafka topic streamed directly into your S3 data lakehouse as an Iceberg table could cost you... $10/hr? Bufstream does it. It's literally too good to be true. I spent 20 hours researching them. Here's their story (2 minute read) 🧵

Can you tell the difference between the end-to-end latencies of a multi-region Bufstream cluster and a single-region one? On the left is the single-region cluster from our last blog post. On the right is a new test we just completed with 100 GiB/s writes replicated through two

TL;DR: Don’t rename fields. Even though there are a slim number of cases where you can get away with it, it’s rarely worth doing, and is a potential source of bugs. buf.build/blog/totw-1-fi…

TL;DR: Compression is everywhere: CDNs, HTTP servers, even in RPC frameworks like Connect. This pervasiveness means that wire size tradeoffs matter less than they used to twenty years ago, when Protobuf was designed. buf.build/blog/totw-2-co…

TL;DR: enums inherit some unfortunate behaviors from C++. Use the Buf lint rules ENUM_VALUE_PREFIX and ENUM_ZERO_VALUE_SUFFIX  to avoid this problem (they’re part of the DEFAULT category). buf.build/blog/totw-3-en…

TL;DR: Protobuf’s distributed nature introduces evolution risks that make it hard to fix some types of mistakes. Sometimes the best thing to do is to just let it be. buf.build/blog/totw-4-ac…

Engineers shouldn’t have to define their network APIs in OpenAPI or Protobuf, their streaming data types in Avro, and their data lake schemas in SQL. A unified schema approach dramatically reshapes our world, solving data quality problems at the source. buf.build/blog/kafka-sch…

TL;DR: Avoid import public and import weak. The Buf lint rules IMPORT_NO_PUBLIC and IMPORT_NO_WEAK enforce this for you by default. buf.build/blog/totw-5-av…

Learn about Bufstream, our Kafka-compatible #streamingdata platform. This workshop will show what’s possible when you understand the shape of your streaming data. Thursday, May 29  9 AM PDT/12 PM EDT/5 PM BST buf.build/events/worksho…

We’re excited to announce that Bufstream is now available on the AWS Marketplace. Enterprise customers can purchase through their AWS account and accelerate their deployment of Bufstream. buf.build/blog/bufstream…

TL;DR: Enum values can have aliases. This feature is poorly designed and shouldn’t be used. The ENUM_NO_ALLOW_ALIAS Buf lint rule prevents you from using them by default. buf.build/blog/totw-6-da…

Don’t miss our interactive workshop on bringing schema-driven governance to Kafka. Our engineering and product teams will deep dive into how it’s done. Plus, we’ll answer your questions throughout. 📷 Thursday, May 29 📷 9 AM PDT / 12 PM EDT / 5 PM BST buf.build/events/worksho…

TL;DR: buf convert is a powerful tool for examining wire format dumps, by converting them to JSON and using existing JSON analysis tooling. protoscope can be used for lower-level analysis, such debugging messages that have been corrupted. buf.build/blog/totw-7-sc…