Data Engineering Industry Report

Data Engineering Industry Report

Our team analyzed 3.5B data points to uncover what the most popular data engineer tools are, what tools companies are spending money on, and what data engineering jobs pay the most. Read the study here.

Here are some of our biggest findings:

  1. The 5 most popular data engineering tools are MongoDB, Tableau, Kubernetes, PostgreSQL, and Ansible.
  2. There are over 340,000 job openings right now in the data engineering field.
  3. Kubernetes, followed by Elasticsearch, PostgreSQL, and Terraform are the highest paying specialties.
  4. Companies are opting to use free data processing tools, like Apache Spark and Apache Kafka.
  5. The industry is split over whether to pay for data storage, with the top four tools being an equal split of paid and free tools.
  6. There are clear winners in the data orchestration space. Kubernetes and Ansible control 65% of the data orchestration space.
  7. Companies prefer to pay for data orchestration tools, with Tableau and Microsoft Power BI holding most of the market.

Click to see all of the result of the study.

要查看或添加评论,请登录

Maria Hatfield, PhD的更多文章

  • Support for OpenSearch & Kafka

    Support for OpenSearch & Kafka

    Many companies use both OpenSearch and Apache Kafka. Both technologies require expert management and 24x7 uptime…

    1 条评论
  • Support for Elasticsearch & Kafka

    Support for Elasticsearch & Kafka

    Many companies use both Elasticsearch and Apache Kafka. Both technologies require expert management and 24x7 uptime…

  • Support for OpenSearch & Pulsar

    Support for OpenSearch & Pulsar

    Many companies use both OpenSearch and Apache Pulsar. Both technologies require expert management and 24x7 uptime…

  • Migrating off of Legacy IT Infrastructure

    Migrating off of Legacy IT Infrastructure

    On our blog today we are walking through a case study where we migrated a client off of their legacy IT infrastructure.…

    1 条评论
  • Increase Kafka Messages per Second

    Increase Kafka Messages per Second

    On our blog today we are discussing increasing Kafka throughput in an environment with network latency. In the article…

  • Comparing OpenSearch and Elasticsearch

    Comparing OpenSearch and Elasticsearch

    On our blog today we are comparing OpenSearch and Elasticsearch. Link to the full post.

  • How to Migrate to OpenSearch

    How to Migrate to OpenSearch

    On our blog this week we are discussing how to plan for a migration to OpenSearch. There are plenty of articles that…

    3 条评论
  • 5 Tips for Reducing Data Storage Costs

    5 Tips for Reducing Data Storage Costs

    With companies revisiting budgets, now is the time to review data storage fees and software licenses to find ways to…

  • Issues to Anticipate When Running Kafka on Kubernetes

    Issues to Anticipate When Running Kafka on Kubernetes

    On the blog this week we've been discussing the challenges presented when running Kafka within Kubernetes. Even when…

  • 5 Optimizations for Kafka Consumers

    5 Optimizations for Kafka Consumers

    On the blog today we are discussing how to optimize Kafka consumers. In short, the 5 optimizations are: #1 Use…

社区洞察

其他会员也浏览了