The Data Scientist vs. the Data Engineer Reloaded
Michael Spencer
A.I. Writer, researcher and curator - full-time Newsletter publication manager.
The Data Scientist vs. the Data Engineer?
If you enjoy programming, datascience and WFH topics, you can subscribe to Datascience Learning Center?here. I cannot continue to write without tips, patronage and community support.
https://datasciencelearningcenter.substack.com/subscribe
Join 34 other paying subscribers. (the price of a cheap coffee)
Data Engineer gains more hype vs. Data Science in 2020s.
Hey Guys,
I’m not a software engineer but the debate of Data Scientist vs. Data Engineer rages on. How would you summarize this debate? I wanted to sort of answer some of the FAQs on this topic. I hope this summary is helpful to someone out there reading this.
But think about it, careers within the field of data science have in recent years seen soaring demand, with the Bureau of Labor Statistics forecasting a 22% increase in job growth from 2020-2030—much higher than the average growth of other occupations.
Which is better data science or data engineer?
Simply put, the data scientist can interpret data only after receiving it in an appropriate format. The data engineer's job is to get the data to the data scientist. Thus, as of now,?data engineers are more in demand than data scientists?because tools cannot perform the tasks of a data engineer.
In the recent past, the general belief in the industry was that as more and more advanced automation tools are developed, the need for pure data scientists would erode. But that hasn’t played out (yet) and may not.
Data Engineers Earn More
What pays more data engineer or data scientist?
Data engineering does not garner the same amount of media attention when compared to data scientists, yet their?average salary tends to be higher than the data scientist?average: $137,000 (data engineer) vs. $121,000 (data scientist).
You do the math, over a career that’s a significant difference.
Data science is easier to learn than data engineering.
Why? Well there's simply more resources available for data science, and there are a number of tools and libraries that have been built to make data science easier.
It’s all a bit confusing as these titles are different at different organizations, for instance:
Can a data scientist become a data engineer?
At some organizations, data scientists are tasked with doing things that data engineers should. While?data scientists aren't equipped with the skills to become data engineers, they can acquire the skills. On the other hand, it's far less common when data engineers begin doing data science.
Job Descriptions are Different
Today, the main difference between these two data professionals is that data engineers build and maintain the systems and structures that store, extract, and organize data, while data scientists analyze that data to predict trends, glean business insights, and answer questions that are relevant to the organization.
领英推荐
That is, Data scientists build and train predictive models using data after it’s been cleaned, and then they communicate their analysis to managers and executives. Data engineers build and maintain the systems that allow data scientists to access and interpret data. The role generally involves creating data models, building data pipelines and overseeing ETL (extract, transform, load).
That’s not to say that data scientists aren’t technical, they just aren’t only working on Engineering.
Why data engineer is better than data scientist?
Data Engineers collect relevant Data. They move and transform this Data into “pipelines” for the Data Science team. They could use programming languages such as Java, Scala, C++ or Python depending on their task. Data Scientists analyze, test, aggregate, optimize the data and present it for the company.
So it all depends in the workflow where you prefer to be.
The science part of Datascience might not appeal to everyone:
As part of their job, they conduct online experiments, develop hypotheses, and use their knowledge of statistics, data analytics, data visualization, and machine learning algorithms to identify trends and create forecasts for the business.
While data engineers are really knee deep in the nitty gritty.
Does data science require coding?
All jobs in Data Science require some degree of coding?and experience with technical tools and technologies. To summarize: Data Engineer: Moderate amount of Python, more knowledge of SQL and optional but preferable is knowledge on a Cloud Platform.
The past five years we’ve been trying to decode the difference between Data Science and Data Engineering and it may still in 2022 depend on the company, industry and the needs of the moment.
Many data engineers and data scientists hold a bachelor’s degree in computer science or a related field such as mathematics, statistics, economics, or information technology.
But think about it, with the increasing integration of AI and machine learning in data analytics platforms, the data scientist of tomorrow may no longer need to have degrees in quantitative fields or to develop algorithms from scratch. What do you think?
Data Science Still Out earns MBAs
Who earns more MBA or data scientist?
The recent placement data from Symbiosis Pune reflects that a postgraduate program in?Data Science?when compared to a general MBA degree has better placement opportunities in terms of average salary and highest package offered.
Data Engineering is still a tough sport and is considered a stressful job:
Is data engineering stressful?
Many factors force data engineers to work long, irregular schedules that take a toll on their well-being. In fact,?78% of survey respondents wish their job came with a therapist to help manage work-related stress.
A?Dataquest blog?explains that the data engineer usually lays the groundwork for the data scientist to “analyze and visualize data.” Some of the initial tasks performed by the data engineer may include managing data sources, managing databases, and launching tools to make the data scientist’s job easy. So, strictly speaking, the data engineer handles all the back-end tasks of data analytics that lay hidden from the public eye.
Different Types of Data Engineers
Read the full article here.
If you enjoy programming, datascience and WFH topics, you can subscribe to Datascience Learning Center?here. I cannot continue to write without tips, patronage and community support.
https://datasciencelearningcenter.substack.com/subscribe
Join 34 other paying subscribers. (the price of a cheap coffee)
Technology Governance Firefighter
2 年The job with the most long-term security will be those that maintain the integrity of the data pipelines. There is much talk about what we do with data, but little emphasis is given to how important the collection, cleaning, and maintenance of data is for those trying to make accurate predictions with it. This type of role will only grow as the data footprint increases.
Strategic Planning | Consulting | FP&A | Project Management | Performance Management
2 年Insightful. Thanks for sharing, Michael Spencer.
Linux Systems Administrator / DevOps / Webhosting Specialist
2 年The idea of engineering being in a contest with science is kind of comical
This was a great read! I appreciate the time you took to clear up a few questions about data scientists vs. data engineers.
Vice President, Semantech Inc.
2 年Interesting article, thanks for posting. I've followed and sometimes participated in these types of debates over the past couple of decades - sometimes they prove meaningful - other times not so much. In this case, I think the true divide between the roles might be a Business vs. IT focus. In other words, Data Science seems to blend well with data SMEs coming at things from a business perspective and engineers are typically more focused on the overall solution. That doesn't mean there isn't overlap - there certainly is. I think the engineering side might be little harder (speaking from experience) simply because the solution designers have to accommodate whatever type of data exploitation is required (and that means being both business agnostic in general and business experts when necessary). In other words, it would probably be easier for an engineer to shift business / functional domains than it would be for a Data Scientist (who may more likely be an expert in specific business process domains). In the end, both roles are complementary and necessary in most organizations.