66% say AWS is the most required platform in job descriptions

I regularly do surveys on my LinkedIn with thousands of votes. It helps me to get a sense of how things are developing in the data engineering world. And on top of that, I get a better understanding of my community and can exchange ideas. One topic I dive into regularly are Data Platforms. This is what I found out about one of the most important parts of Data Engineering.

The Top Clouds

It’s more or less undisputed that AWS is the most used data platform out there. People also see this in job descriptions where AWS has a major lead with 66% over Azure and GCP.

AWS is also perceived as the best platform for Machine Learning.

What’s interesting here is the jump for GCP from 9% seen in descriptions to 26% votes for Machine Learning. Is this because of marketing or is GCP really that good? Let’s see in the following.

Stream vs. Batch Processing

I was wondering what people use most: stream or batch processing? So I ran these two surveys. The poll “With which systems are you working more often?” shows that 81% work more on the data warehouse side and 19% on the transactional side of the data platform. Which translates to 81% working on the analytics / business analytics side of a platform and 19% working more on the business driven side (transactional) of the platform.

The result for stream or batch processing backs this up with almost exactly the same numbers as 83% work most on batch processing (ETL jobs) and only 17% on streaming. So, if you just start your engineering journey, look into ETL and data warehouses.

This ratio also explains the popularity of GCP for ML compared to AWS. With BigQuery and the ML products Google has a strong, if not the best offer on their platform.

Let me know in the comments what your thoughts are regarding these topics. Do you agree with the majority or do you have any other favorites? I’m very curious for your opinion!

Andreas

PS:

In any case, it shows well that it is definitely worth starting and working with AWS. I highly recommend having a look at my other Medium blog post where I introduced a?great AWS project?that I teach in my Academy. It is perfect for anyone who wants to start out with AWS and gain a bit of hands-on experience!

If you want to learn Data Engineering then visit my Data Engineering?Academy.

Are you looking for more information and content on Data Engineering? Then check out my other blog posts, videos and more on?Medium,?YouTube?and?LinkedIn!