I am very excited to share that I am joining #apachearrow as committer! Arrow is an in-memory format and libraries for data analytics & data science. It has been a great experience to work with Andy Grove Andrew Lamb @jorgecarleitao and the rest of the team

I wrote a bit about increasing the amount of parallelism in DataFusion: medium.com/@danilheres/in…

DataFusion 5 (an open source query engine in Rust) has been released. Some nice performance improvements made it into this release which can be seen in the benchmark results. arrow.apache.org/blog/2021/08/1…

We've released ApacheArrow 6.0.0, one of the largest and most impactful releases yet! Read more for what's new and look out for more blog posts about new features in the coming weeks and months: arrow.apache.org/blog/2021/11/0…

If anyone is at the DataBricks Data+AI summit, Däniel Heres and I will be presenting on Apache Arrow and DataFusion today at 5:30PM: databricks.com/dataaisummit/s…

Optimizing TopK queries in DataFusion godatadriven.com/blog/optimizin…

Apache Arrow system, Parquet, Datafusion are incredibly strong foundation for the next chapter in data. DuckDB is a delight as well. We are going towards much simpler stacks based on extremely well designed infrastructure and that's really becoming for the data world.

DataPrime from Coralogix is here with powerful features that let you explore, transform, and extract data like never before! Find out how it works here. 👇🏼 Get in touch with me to learn more about how DataPrime can help your team out! #Datasecurity bit.ly/3ZAl9mu

Aggregating data (summing, counting, averaging, etc.) millions of groups got much faster in DataFusion 28. It was fun to work together with Andrew Lamb, Raphael Taylor-Davies, others to make this happen. This article shows how it works and what it means: influxdata.com/blog/aggregati…

.

There is a cool opportunity in my remote team at Coralogix for a Software Engineer to contribute to an ambitious distributed query engine based on DataFusion / Ballista. The job is open for US candidates as well. Feel free to send me a DM for questions. coralogix.com/careers/co/rem…

I am pretty excited about the growth in projects built with ApacheArrow DataFusion. Here is the official announcement of Comet, created by Engineers from Apple to accelerate Spark. arrow.apache.org/blog/2024/03/0…

We keep improving the Rust parquet readers performance. This time with SIMD accelerated UTF8 valdation: github.com/apache/arrow-r… (thanks to Daniël Heres @[email protected] and many others for helping test!)

Why am I not seeing more outcry on the brutal assault on DEI in the US? The queer community is the first target of authoritarian regimes (or worse). This is the time to show your support even if you feel pressured not to (in which case you have already lost freedom of speech). 🏳️‍🌈

Thank you for your support! 🇺🇦 __ Дякую вам за підтримку! 🇺🇦

DataFusion 45 Release Blog: datafusion.apache.org/blog/2025/02/2…

I had the privilege of presenting Hudi-rs at VeloxCon 2025 - the native Rust implementation of Apache Hudi that's opening new doors for cross-language data processing. 🔨 By rebuilding Hudi's core in Rust, we've created a foundation that supports multiple language bindings

Example of the kind of low level obsession we foster in the Rust parquet / arrow / DataFusion community: github.com/apache/arrow-r… I am skeptical that proprietary engines will be able to compete with OSS long term (though I am biased) Huge thanks to Qi Zhu and Daniël Heres @[email protected]