Josh Wills Abhi Sivasailam Ananth Packkildurai john kutay all of data engineering is duct tape around the impedance mismatch between software engineering and analytics

This is incredible.

From Jamie Lane where AirBnBs are fully booked next week.

SF remains an amazing controlled experiment between the strength of an economic network effect and an attempt by a governing authority to destroy it.

The Dagster GitHub repo has officially reached 10,000 stars!

We appreciate our growing community + the collective effort of the contributors, users, and supporters who believe in making data orchestration more approachable, reliable, and productive.

Here's to the next 10K.

For All Mankind is a demented expansion pack for Twilight Struggle in tv show form.

In our latest redesign of Dagster's Asset Catalog, we've focused on making asset metadata more accessible from the UI. You can now easily find details like column schema, materialization metadata, asset status, description, lineage, and more — all located in a single spot.

When does the Huberman thing get a Sankey diagram

Column-level lineage 🤝 Dagster

➕

Congrats to Pete Soderling and team for raising their second fund! Super knowledgeable team that has fostered a great community of founders.

imagine seeing this and not feeling incredible wonder about the scale of the universe

Watch Starship’s third flight test → spacex.com/launches twitter.com/i/broadcasts/1…

I had a great discussion with Johannes on his podcast about local-first/sync engines.

Sync engines eliminate an entire class of problems around manual data fetching. Similar to how jQuery -> React freed us from manual view rendering.

Also on the blog: bricolage.io/localfirst.fm-…

We promised you a Monday meetup.
The wait is almost over.

Join us in 4 days in NYC with Oz Katz from lakeFS and Nick Schrock from Dagster discuss troubleshooting and reproduction strategies to ensure high quality data in production.

Register now:
bit.ly/48Kca6n

📢 The Data Engineering track at Data Council Austin '24 is going to be 🔥 🔥 🔥
Check out the lineup! 🤩 👀 👇

🔥 Nick Schrock Nick Schrock | Founder & CTO | Dagster
Declarative Orchestration: Why You’ve Been Thinking About the Wrong DAG the Entire Time

(1 of 7)👇

Could not be more excited to start at Dagster as a sales engineer!

Why Dagster? Dagster makes an OSS and commercial data orchestrator. As Nick Schrock says, data orchestration powers a huge amount of decisions that get made every day (mortgage? Health insurance? Pipelines all)

What a take on Gemini by 𝖦𝗋𝗂𝗆𝖾𝗌 ⏳ . Few people capable of this sort of insight and eloquence. She’s right: this is a performance art masterpiece.

A massive step forward for the entire Python ecosystem. Charlie Marsh is coming to save us all.

Belief I have about data platforms: they should be monolithic.

I.e one per organization, not one per team.

This means putting in work to:
- Standardize how pipelines are written and how data is exposed
- Unify the set of tools for control and observability

But it's worth it:

Absolutely massive improvement to our UX in this release. Asset graph now effectively infinitely scales and is a better experience. Yes and we added dark mode.