Important stuff here 👇 It's great to see this transformation of the data stack gaining steam (and raising more capital) but much work still remains

New blog post — Voltron Data Update: Transitions wesmckinney.com/blog/voltron-d…

Overpublishing puts enormous stress on students and PIs. And brings tons of money to publishers in STEM. A new study shows that the number of papers is increasing FASTER than the number of #PhD graduates. It’s an amazing work with very useful statistics. Huge kudos to the

Last month the "DB Research Meeting" was held @mitcsail, hosted by Sam Madden and Natassa Ailamaki (🙏). An encounter of the who's who in data systems research. I warned for the declining impact of DB research & pitched better incentives for system work: bit.ly/dbmeeting-boncz

I'm back again with my annual retrospective of the last year in the world of databases. Major highlights include vector databases, @MariaDB problems, SQL:2023, the FAA database crash, and the most expensive password change ever: ottertune.com/blog/2023-data…

"I had this crazy idea that I’m going to build a database engine that does not have a server, that talks directly to disk, and ignores the data types, and if you asked any of the experts of the day, they would say, “That’s impossible. That will never work. That’s a stupid idea.”

Go Badgers!

CfP: Data Management on New Hardware Workshop, co-located with ACM SIGMOD/PODS in Santiago, Chile. Papers due: March 15, 2024. Carsten Binnig and I are looking forward to your submissions to this *** 20th special edition of DaMoN ***! #DAMON2024 #SIGMOD2024 damon-db.org

The SIGMOD Jim Gray Dissertation Award has started receiving nominations! Deadline: March 15th, 2024. For more information please visit sigmod.org/sigmod-awards/…

My #1 PhD student @butro successfully completed his PhD defense. Thanks to the committee (Jignesh Patel billions of packets Sam Madden). Matt's thesis is on accelerating databases with eBPF (telemetry, proxies, OLTP stores). You have 60 days to hire him. Expect fierce competition.

Somebody tipped me off that a 2022 paper out of Saudia Arabia blatantly stole our entire 2019 ICDE Bulletin survey paper on using ML automatically optimize databases. + 2022 Plagiarism: eajournals.org/ejcsit/vol10-i… + 2019 Original: db.cs.cmu.edu/papers/2019/pa…

Columnar file formats like Parquet/ORC are ubiquitous. Our VLDB paper with Xinyu Zeng + Huanchen Zhang + Wes McKinney studies their internals. TLDR: They're not optimized for modern hardware. Something new is needed. Paper: vldb.org/pvldb/vol17/p1… Code: github.com/XinyuZeng/Eval…

CedarDB: The 🇩🇪German-powered, PostgreSQL-compatible freak-of-nature database management system based on TUM's Umbra (Thomas Neumann + team) is out of stealth and now available: cedardb.com/blog/ode_to_po… /cc CedarDB

My group CWI DA is once again doing the local organization of my favorite conference: CIDR 2025 Check its exciting program here: cidrdb.org/cidr2025/progr… It will be held January 19-22 in the Amsterdam Mövenpick. Plan your trip quickly, because registration closes this Thursday!

CIDR2025 is a wrap. Loved the talks & audience questions, Gong Show, DuckDB reception..⁦ Association for Computing Machinery⁩ pres Yannis Ioannidis talked on open science. Proceedings are in ACM DL & VLDB (see cidrdb.org). 🙏 all in+outside CWI DA⁩ who helped organize!!

Join ordering may not be a critical challenge for future optimizers anymore! Check out our latest paper on robust query processing (to appear in SIGMOD'25): arxiv.org/pdf/2502.15181 Xiangyao Yu Andy Pavlo (@andypavlo.bsky.social) Jignesh Patel Peter Boncz Yuanyuan Tian

Insightful post on why Apache Iceberg may not be a one-size-fits-all solution when it comes to a table format to manage large multimodal ML/AI datasets

Vol:18 No:9 → Concurrency Control as a Service vldb.org/pvldb/vol18/p2…

Vol:18 No:11 → Selective Late Materialization in Modern Analytical Databases vldb.org/pvldb/vol18/p4…

Our SIGMOD paper with Xinyu Zeng + Huanchen Zhang + Wes McKinney + Jignesh Patel on creating a next generation open-source data file format is out. F3 is a future-proof file format avoids the mistakes of Parquet. 📄 Paper: db.cs.cmu.edu/papers/2025/ze… 📁 Code: github.com/future-file-fo…