An executive’s career advice for data scientists - Part II.

Addressing the mismatch between data science aspirations and company realities

Reality vs. Expectations

A few weeks ago, I wrote about the three main structural challenges that data scientists and their organizations face when it comes to maximizing their career satisfaction and business impact:

1.??????Mismatch between aspirations and reality

2.??????Focusing on solutions, not problems

3.??????The CXO “AI/ML fallacy”

This week we’ll dive deeper into the first impediment (“aspirations vs. reality”), decompose why it exists, and make some recommendations for both data scientists and their companies on how best to address it.

Most innovations go through a natural hype cycle over the course of 10-15 years, starting with a trigger, leading to a peak (of inflated expectations), dipping to a trough (of disillusionment), before normalizing into a plateau of productivity. For data science and AI, we’ve been at the peak for about a decade, and for most companies in non-tech industries, the AI train is still going uphill.?

Organizations are investing tens of millions of dollars into industry-disrupting analytics transformations, with about 40% pouring into digital transformation/acceleration type consulting engagements and 60% into premium-prized human and tech capital. Analytics, data science, AI and ML are key items on the CXO agenda, and Chief Analytics/Data Officers or Chief Data Scientists are increasingly getting a seat at the table for executive leadership and board conversations. Data scientists are one of the highest individual contributor corporate earners, have enjoyed a 40-60% job growth over the last 10 years, with a 30% annual job growth forecasted through 2026 .

The data nerd in me has enjoyed seeing all the love and investment data science has received, and it gives me encouragement that data and analytics will continue to enjoy healthy levels of human and capital inflows. The altruistic side of me also loves reading inspiring career success stories and seeing data science education be a great equalizer for those from less advantaged socioeconomic backgrounds.

However, having occupied key leadership roles in analytics transformations, and interacted with many data science teams both inside and outside my industries, I recognize that there is a wide gap between data scientist aspirations and the realities in which companies operate, driven by three key reasons.

1. Corporate stakeholders don’t really know what data science can or should do

The Problem

If you ask the heads of Sales, Marketing, Finance, IT and Supply Chain what data science is or how a data science team can help their functions, you often get five different answers. Is data science just stats? Is it machine learning or actually AI? Is it about descriptive analytics and nice dashboards? Perhaps someone who can do custom data pulls in SQL very fast?

Unless the analytics roadmap is co-created with the above CXOs and their 2nd and 3rd level team leaders and widely proselytized throughout the organization, there will continue to be lack of alignment that will result in frustrated parties on both ends. I.e., the VP of Sales asking a Sr. Data Scientist for something mundane or the Sr. Data Scientist delivering a machine learning model when a simple correlation matrix would have done it.

Advice for the data science team and business leadership

Establish the rules of engagement early on between your data science groups and functional stakeholders. Create an analytics or data science steering committee comprised of the CEO and her functional leadership team – meet once a month on project updates, critical use case prioritizations and investment asks.

Proselytization is important, especially in the first 12 months of the data science journey, therefore analytics leaders need to partner with CXOs to drive the advanced analytics agenda. Set up bi-weekly or monthly informal gatherings (i.e., lunch & learns, analytics meetups) to educate non-data science colleagues about data science in general, and more specifically about ongoing data science initiatives in the company (use the right language and avoid complexity).

Ideally partner with the executive leadership team to identify one member per key business function as a Sr. Manager to Sr. Director level “Analytics Translator” and 2-3 members as “citizen data scientists”. The former will be the most important link between the data science team and the business units while the latter will ensure that most descriptive, diagnostic and essential predictive analytics requests to Sr. Analyst to Sr. Manager level folks in the BU who are domain experts and are familiar with foundational machine learning and have strong self-serve, democratized data visualization and data engineering skills (e.g.: Tableau, Power BI, Alteryx, with some basic R or Python).

Companies need to get their data infrastructure in order first (especially operational, financial and customer data), and wherever appropriate, create purpose-built analytics data warehouses in AWS, Azure, GCP, etc. that are optimized for speed and scale. 60% of data scientists are still spending majority of their time cleaning and migrating data from some archaic on-prem data warehouse into a more suitable infrastructure. That’s a lot of time that could be used for generating actionable business insights, doing feature engineering or piloting and deploying models.

2. Bifurcated Analytics Leadership Skills

The Problem

Data science unicorns are rare (those that excel at the trifecta of advanced machine learning theory and practicum, data storytelling and stakeholder management), and so are data science leaders that exhibit similar skillsets. Executives in charge of advanced analytics or data science teams are highly bifurcated. They are either strong businesspeople, domain experts, team managers, corporate cheerleaders and stakeholder influencers, OR are strong technical folks with superb data science foundations – akin to a Chief Data Scientist. The former cohort often gets promoted internally from Finance, Marketing, Product or IT, while the latter usually have backgrounds in academia or research-oriented data science. It is hard to find analytics leadership talent who encompasses elements of both (business/domain + data science), yet that is exactly what your organization and your data science team needs to succeed and deliver measurable outcomes.

Until things mature and you have a solid data science team with 12-18 months of domain experience, your team members will need guidance on the appropriate analytics and modeling methods for certain business problems. They will need help framing the problem in a way that makes sense to them, and similarly they will need help with translating the complex machine learning methods into simple business speak that senior executives will understand. Having someone with the blend of ML and business who understands the data science workflow and how to turn data into measurable customer outcomes will make the data scientist experience much stronger and the overall analytics team more effective.

Advice for the data science team and business leadership

Instead of hiring the data science or analytics leader at the ones, hire the first 2-3 data scientists and temporarily place them with key business functions. This allows them to slowly become domain experts and hit the exponential curve on value add much sooner. If you are not quite sure who to hire (it’s okay, you are not alone), use an expert consultant for the first 6-12 months to help you build up the team, and to help establish a data science use case pipeline and operating rhythm.

Leverage your existing data science team and other in-house data analytics or engineering experts, in tandem with key business stakeholders to all be part of the analytics leadership interview process. Their synchronized feedback will be more meaningful and ultimately more impactful for the business versus overwhelmingly relying on executive peer and senior leadership interviews. Your data scientists and other quantitative functional leaders in the organization can easily identify the astute corporate operator who otherwise has little to no advanced analytics acumen. And the functional leaders (executive peers) and senior leadership can easily identify the potential data science exec candidate who is overly theoretical or who places too much emphasis on the technology and algorithms versus generating measurable business outcomes.

3. Too much focus on ML techniques vs. soft skills

The Problem

Understanding ML frameworks and strong coding skills in R or Python are the baseline requirement for data scientists. Equally if not more important is the ability to understand and frame business problems, understand how your stakeholders are making decisions, effectively manage your internal customers and to produce models, insights and recommendations that are translated to your audience. Without these soft skills, your analytics project or initiative will not make it into production, will not get used, and business decision makers will certainly not think that you as the data scientist is adding any value.

Advice for the data science team and business leadership

Unlike most people in the analytics world, I’m an MBA-turned-coder who discovered the beautiful world of R and Python during business school. One of the most underestimated and ignored pieces of our graduate school program were almost always our “Organizational Behavior” and “Strategic Communication” classes – taught by two otherwise excellent professors and leaders in their field. Most of us wanted to take advanced finance classes, decision analytics, quantitative marketing courses and the like. It took at least 5-10 years post business school to realize that hardly any of us actually used the “Black-Scholes model” or run a conjoint analysis by hand, but that all of us would have benefitted tremendously from some more intentional and strategic stakeholder management and organizational communication acumen.

The analogy also holds true for your data science teams. Increasingly, advanced methods are being democratized (i.e., PyTorch, Tensorflow, various AutoML solutions, etc.) and this trend will continue. The data science and analytics field will need more people who are structured thinkers, skilled data storytellers and effective communicators, are laser focused on solving important problems, and care about driving measurable customer outcomes (i.e., they are results-focused).

My advice for data scientists is to start carefully honing your competencies in communications, influencing, data storytelling, domain expertise and strategic thinking. Similarly, my advice for companies is to start hiring data science and analytics talent who are strategic, methodical thinkers and effective communicators, with a demonstrated history of driving measurable value for organizations. For more novice roles, focus on hiring talent who show a healthy balance between the soft skills and the hard technical and analytical acumen, and spend the right resources to upskill them in the right competencies.?