Is it a Data Science Shortage or a Leadership Shortage?

I sometimes feel like I am in the movie Ground Hogs Day. Except it is with how companies run their data science practice. A company hires some sharp person who can code well enough and they hand the keys over to this person to “build the future.” Months, or years later, the works is a hot mess and fingers are being pointed left and right as deadlines are not met. What happened? The company didn’t account for the fact they need a data leader, not just data scientists. 

It seems every company has a data scientist these days. Even some company in the middle of cornfields in the Midwestern US has a data scientist. However, most of these companies are failing at getting these data scientists to produce anything of real substance for the company. Some how they believe that data scientists will magically figure out how to run an efficient business team without any really guidance or leadership. Then they end up like one company I know that has been working on a pricing model for 4 years and still hasn’t gotten it to work without any deadline to do so. Why? No real leaders running the team.

Data Science or Data Leadership

I often get to talk to recruiters who either would like me to join their client’s team or are asking me to help them find someone. I am happy to talk. The roles we talk about when it comes to senior level, often have the “they want a hands-on leader” thrown in. I have to always ask, what does that mean? If I get the response, “someone who can code”, I politely let them know that it is not a leader they want but a senior data scientist which isn’t the same as a data leader. 

If you want someone to code, that’s fine, but be honest, you want a coder. A leader doesn’t do much coding. Not because the person can’t, it is because there are a lot of other things that need to get done and often that leaders is the only one who can do it. Who is managing the patent strategy, legal strategy, P&L, environments requirements, staffing requirements, business case alignment, corporate strategy alignment, roadmap (the real kind, not that sprint schedule some call a roadmap). Often there isn’t anyone on staff that understands data well enough to take these tasks on, yet they are more and more important as times go by. So why would you want someone coding all day, so you can get in trouble because you didn’t do patent searches to ensure you weren’t violating an existing patent? Or maybe you were really hoping to violate GDPR this month? Or you wanted a coder so nobody was really focusing on why you doing that coding in the first place? Big difference here. One that many companies find out the hard way.

Set Up a Data Practice, not a Data Science Team

Most companies are still struggling to understand data. They may understand software, hardware, physical goods but that doesn’t often translate into understanding data and the language of data. Why is relevancy so important? What is a filter bubble? How about a heterotopia in regards to data? Most execs can’t answer these questions and most data scientists can’t either without looking it up. And even fewer can actually put these concepts to use to drive business growth. Add to that, a data practice is about changing the culture of an organization, which requires skills beyond math and coding, lots of skills beyond math and coding. If your team isn’t achieving objectives, it is because you don’t have a data leader.

A data leader is about building the right data culture. One focused on the business objects and around improving the customers experience (b2B and B2C) through the use of data. This involves way more than coding, in fact I would say coding is the easiest and least important issue. There are legal and compliance issues you will face. Ensuring the team is not violating patents, customer privacy or any regulations with the development of their algorithms is a very important part of the job. You can’t do that if you just code. A data leader needs to be able to review the code and models but spends a lot of time having to review patents and regulatory issues, something most data scientists spend about zero percent of their time on.

A data leader needs to focus on the flow of data and performance. I know a ton of data scientists who think they know this part but truly it is an engineering practice. A data leader needs to understand this and make decisions around buy vs build on a daily basis. Focusing on the performance of the platform and how viable will it be for the life of the data products being built. I have had plenty of data scientist tell me they can do this job, only to turn around and admit they are out of their league. In fact, one time, one wanted to us AWS and told me it would only cost $1,600 a month. Turns out the bill was 15X that, because he didn’t know how to ask for the cost of the actual work. Luckily, we stopped that before it happened.

A data leader needs to create a learning culture. Again, plenty of data scientists say they know how to do this and for themselves I would agree, they do. But you don’t create an educated team of one. You need a culture of learners. Setting up a structure that allows for learning and constant improvement of one’s skills is a critical role for a data leader.

A data leader needs to set up processes that are appropriate for the team’s growth. As a team grows and takes on more responsibilities, they have to go from a start up like team to a more mature team. How to market responds to their products also changes and a good data leader understands this lifecycle and builds solids processes for the stages of the team. I have seen many a data scientists fail big time because they did not understand the growth phases they need to shepherd their team through.

A data leader understands that data teams are more than just engineers and data scientists. I see this all the time, a team building a recommender and they don’t have anyone from UX on the team. They don’t use personas and if they do they don’t really appreciate them. Huge mistake. My basic data teams always have UX and my more complex established teams have social science skills, business skills, legal skills, offline team member skills, etc… Why? Data is often an expression of human activity. Without that understanding of the human, you don’t really know your data. 

Data leaders are very important to the success of your data science practice, yet most companies are not hiring data leaders, just data scientists who can code. Last year I saw a couple of studies stating that 80-90% of data projects fail to meet a business objective. Why is that? In my opinion, they didn’t have a data leader running the team. If you want success with your data practice, get a data leader. The difference is huge. Here’s an example:

I worked for a client who had spent two years working on a data product. They had a number of data scientists, many with PhD’s. However, the project was not delivering, $10MM spent and no idea when the project would be completed. I was called in. Right away I organized the team in a manner that would stream line the work. We set up deadlines based on abilities of the team. Review of the code was set up to ensure it would not violate any patents or laws. Test environments were streamlines to go from Proof of life to production as quickly as possible without sacrificing necessary testing. And financial projections and business goals were set up. 

Six month later, that product launched. Minor rework needed as up keep and what was to lose another $10MM in costs this year is now on target to hit $155MM in revenue this year with a healthy ROI. That’s what a good data leader does for a company. 

Want to know more, feel free to connect with me.