' #Snowflake Integration Patterns: #ZeroETL and #ReverseETL vs. #ApacheKafka '

=> Back home in Germany after a two-week US trip, it is time for a long awaited blog series... Here is part 1:

kai-waehner.de/blog/2024/04/1…

'Upgrading #DataWarehouse Infrastructure at #Airbnb with Spark + #Iceberg ' => The data ingestion framework processes >35 billion #ApacheKafka event messages and 1,000+ tables per day.

50% compute resource-saving + 40% job elapsed time reduction.

medium.com/airbnb-enginee…

'The state of #datastreaming for #healthcare '
=> Industry use cases and architectures for #apachekafka and #apacheflink .

I look at trends and customer stories from Humana, Recursion, BHG (former Bankers Healthcare Group), and more.

kai-waehner.de/blog/2023/11/2…

#ApacheKafka and #ApacheFlink are increasingly joining forces to build innovative real-time #streamprocessing applications.

But When to use ' #KafkaStreams vs. Flink for #datastreaming ?

kai-waehner.de/blog/2023/01/2…

#opensource

'Data Streaming with #ApacheKafka for National Security and Defense'

=> Exciting use cases with #hybrid and #edge deployments of #kafka for #nationalsecurity , #cybersecurity and #defense in the #publicsector .

kai-waehner.de/blog/2021/10/2…

The problem with trying to sell developer tooling is that developers have no purchasing authority

Salesperson needs to spend $1000? No big deal.
Finance needs to spend $100,000? No big deal.
Engineer wants to buy a $50 book? They need forms signed from their VP in triplicate.

'A Real-Time #SupplyChain Control Tower powered by #ApacheKafka '

=> A modern supply chain requires just-in-time production, global #logistics , and complex #manufacturing processes.

kai-waehner.de/blog/2022/09/2…

'Keeping Multiple #Databases in Sync in Real-Time Using #ApacheKafka Connect and #ChangeDataCapture '

This blog post will review the advantages and disadvantages inherent to moving data from a database using #KafkaConnect , #JDBC and #CDC

confluent.io/blog/sync-data…

Better late than never… #Google announced a brand new #ApacheKafka cloud service for GCP today at #googlecloudnext ! All other leading #cloud providers already have one, including AWS, Azure, Oracle, IBM, and Alibaba.

More details in my latest blog post

kai-waehner.de/blog/2024/04/1…

'Building a Postmodern #ERP with #ApacheKafka '

=> A Postmodern ERP combines #opensource technologies and proprietary standard software. Many solutions are #cloudnative or even offered as fully-managed #SaaS cloud offerings powered by Kafka.

kai-waehner.de/blog/2020/11/2…

'Getting Started with the #KRaft Protocol for #zookeeper removal from #apachekafka '

=> ZooKeeper will be deprecated in Kafka later this year! Better plan upgrading soon :-)

confluent.io/blog/what-is-k…

#datastreaming #opensource #cloud

'The Daily Life of a #FieldCTO '

=> A question I get every week: What does a Field CTO do? This is a report about my #dailylife . You might learn a few things, or just enjoy reading about a relatively new #job category for your #career options...

kai-waehner.de/blog/2023/01/0…

'Data Streaming with #ApacheKafka and #ApacheFlink as Data Fabric for #GenerativeAI '

Example of an enterprise architecture leveraging event-driven data streaming for data ingestion and processing across the entire #GenAI pipeline:

#BigCommerce is a #cloud native SaaS #eCommerce Platform enabling merchants to create B2B and B2C commerce solutions. Really impressive how they use #datastreaming powered by #apachekafka in the #cloud for various use cases, including:

medium.com/bigcommerce-en…

🚨 Breaking news:

YouTube just got a massive upgrade.

AI is now coming to YouTube, and it's insane.

Here's everything you need to know to stay on top: 🧵

#ApacheKafka and Tinybird ( #ClickHouse ) for Streaming Analytics HTTP APIs

=> My latest blog post about a few lessons learned from past customer events with Confluent Cloud and Tinybird... Customer stories: Factorial, FanDuel and Hard Rock Digital

kai-waehner.de/blog/2024/04/0…

'Building Scalable, Real-Time Chat to Improve Customer Experience with #ApacheKafka , #GraphQL and #WebSockets at Uber'

Uber replaced the legacy architecture with a new solution benefiting from GraphQL subscriptions and real-time data infrastructure.

uber.com/en-AU/blog/bui…

#Rimac is well known for luxury cars, but they also sell battery systems, electronic control units, and software. The Rimac Connectivity Platform is built on top of #apachekafka and #mqtt .

Video:

youtube.com/watch?v=-w8w-N…

#automotive #iot #connectedcars #opensource #cloud

'Scaling AI/ML Infrastructure at Uber with Kafka, Flink and Spark'

Uber's engineering blog explores #apachekafka , #apacheflink and #apachespark as the core of the cloud-native data infrastructure for predictive #machinelearning and #GenerativeAI .

uber.com/blog/scaling-a…

' #DisasterRecovery with #ApacheKafka across the #Edge and Hybrid #Cloud '

Apache Kafka is the de facto #datastreaming platform for #analytics AND #transactional workloads. Multiple options exist to design Kafka for resilient applications.

kai-waehner.de/blog/2022/04/0…