I look forward to giving a talk about the General Availability of #ApacheSpark on #Kubernetes at the #DataLove virtual conference on April 16th. Sign up (for a minimum $1 donation) at datalove.konfy.care PS: We're hiring Data Mechanics : angel.co/company/datame…

Excited to share this customer story of Lingk who migrated from #EMR to our platform Data Mechanics The move to #SparkOnKubernetes enabled: * 65% AWS Costs reductions * 2x faster app startup time * 40% average app duration reduction More details: datamechanics.co/blog-post/migr…

We're releasing our optimized #Docker images for #ApacheSpark at Data Mechanics ! Blog: datamechanics.co/blog-post/opti… DockerHub: hub.docker.com/r/datamechanic… They contain Spark, Scala, Java, Hadoop, Python, and connectors to common data sources. We hope to save you from dependency hell!

Serverless #spark platforms sound interesting? We're stoked to be with Jean-Yves Stephan the CEO of Data Mechanics tomorrow. towardsdatascience.com/how-we-built-a…

You don't like the Spark UI? We heard you! We're releasing Delight - a free & cross-platform monitoring dashboard for #ApacheSpark. datamechanics.co/blog-post/deli… Backed by an #opensource agent, It works on top of any Spark platform - Databricks, EMR, Dataproc, CDH/HDP, ...

This is the result of over 6 months of engineering work from our team - check it out and let us know your feedback!

Ep 5 of Deep in data is out! Our very own Robert Lacok spoke to Jean-Yves Stephan from Data Mechanics about Spark, its use cases and how to successfully integrate Spark into your data stack. 👂: open.spotify.com/episode/3DWp7d… 👀: deepnote.com/@Deep-in-data/…

Humbled to be featured in the #DataEngineering Podcast: dataengineeringpodcast.com/data-mechanics… We talked about how we try to make #ApacheSpark #CloudNative at Data Mechanics ! We talked about Delight, our open-source monitoring dashboard for Spark with CPU & Memory Metrics.. and much more!

Data Mechanics now powers the United Nations Global Platform, enabling statisticians around the word to analyse maritime traffic data. Migrating their #ApacheSpark platform from #EMR to #SparkOnKubernetes improved their UX & reduced their costs by 70%! datamechanics.co/blog-post/how-…

Tutorial: Running PySpark inside Docker Containers: datamechanics.co/blog-post/tuto… A couple of months ago we published our Docker images for Apache Spark on DockerHub (they come packaged with common data connectors and libraries). This tutorial shows you how to get started with them!

#ApacheSpark 3.2 was just released and it's now available on Data Mechanics! New Features: * Spark-on-k8s specific: PVC reuse making Spark robust to exec loss by disaggregating compute & storage. * Spark now bundles Koalas, Hadoop 3.3.1, and RocksDB. datamechanics.co/blog-post/apac…

6 months after our acquisition by @spot_by_netapp, we're launching the joint Data Mechanics + Spot product called Ocean for Apache Spark. We make Spark on k8s more developer-friendly and cost-effective. Proud of our global team who made this possible! spot.io/blog/ocean-for…

Since the Spark 3.2 release, the S3 Magic Committer is a lot more easy-to-use, performant, and stable. It enabled a 60% performance boost in our tests! Learn more in this technical blog post about #ApacheSpark performance tuning on #AWS #S3 spot.io/blog/improve-a…

Ocean for Apache Spark is now officially GA on AWS: spot.io/blog/ocean-for… Main benefits of running Spark on k8s with us: intuitive UI, powerful automations enabling big savings, integrations with jupyter & airflow, ... Very proud of our team! GCP & Azure are coming next :)!

In this tutorial we show you how to run Jupyter Notebooks on our serverless Spark-on-k8s offering, Ocean for Apache Spark: spot.io/blog/tutorial-… PS: In addition to Python notebooks, Scala notebook are now in private preview - reach out if you're interested!

Ocean for Apache Spark, our fully managed, continuously optimized, Spark-on-Kubernetes service, is now available on Google Cloud. It's deployed on a GKE cluster in your GCP account. Learn more: spot.io/blog/ocean-for… Congrats to the team! #AWS ✅ #GCP ✅ #Azure: coming next!

Met long-time online friends Jean-Yves Stephan and Julien Dumazert from Data Mechanics finally! 💪 It's the first time when we could give a hug each other! 😉 Merci beaucoup mes amis pour notre bien accueillir ♥️ #DataAISummit

Data + AI Summit is a wrap! If you couldn't attend our session, we just released the same content as a blog post - How to run Spark on Kubernetes reliably on spot instances :) ! spot.io/blog/how-to-ru…