A data scientist, a data analyst, and a business intelligence expert walk into a bar …

What starts as a joke for some, means nothing for most people. Those who are deeply entrenched in the analytics field define these functions as different roles including also positions such as data engineer, decision scientist, or data researcher. Most analytics experts currently prefer “data scientist” as the title describing the most seasoned experts who focus especially on predictive analytics. “Business intelligence” is out of favor since it has been used for a long time to describe the analysis of what has happened in the past. Only two letters separate “insight” and “hindsight”, but apparently these are two completely different worlds for some.

Business people, on the other hand, use these job titles interchangeably; most of the time they are all and the same. And anything with the word “scientist” or “engineer” may imply a data-focused expert who has limited business understanding. For business people, “business intelligence” seems to better describe how to extract insight from data.

All these titles are used so loosely and with so little understanding that is mind boggling, although this may not be unusual for a new frontier such as data analytics. These titles will change as Application Service Providers in the 1990s became Software-As-A-Service in the 2000s, and just Cloud in the 2010s. Before dwelling too much on the names, it’s important to describe what all these people do, so let’s clarify what needs to be done throughout the entire value chain of data analytics.

The data analytics value chain starts with capturing and standardizing data, continues with extracting information, which together with business and market context is used to identify insight, and then is turned into prioritized short- and long-term business decisions:

Data

Data comes in different shapes and forms, whether it’s structured or unstructured, sporadic or continuous data flow, transactional or non-transactional. And most of the time it includes both signal and noise. In all cases, data needs to be captured and cleansed in order to standardize it for future analysis. Data cleansing involves such simple tasks as ensuring that “June” and “Jun” are captured as the same month. Data centralization is a key part of this step to ensure it can be accessed through one central data repository for uncomplicated future analysis. Before extracting information, most of the data sets need to be clustered. Clustering creates distinctive data groupings, groups of customers or groups of products, for example, allowing for an effective analysis in the next stages of the value chain. While most of the tasks needed to manage data rely heavily on data administration skills, clustering requires some business knowledge since algorithmic clustering may not always be feasible without the business context. Business context is especially critical in capturing non-transactional data such as customer surveys. Data from customer surveys may only be qualitative or directional, so it needs to be managed differently than quantitative transactional data.

Information

Extracting information from data starts with answering the question about what has happened. The focus is on the past: how many more products did we sell last month vs. a year ago, have we reached the expected click-through rate in our online marketing campaign, have we gained or lost market share in the consumer segment last quarter? Although extracting information is a relatively simple task where many of its aspects can be automated, it requires business context to separate signal from noise. Algorithms can find patterns in data, but business acumen and domain expertise are needed to find useful patterns in data. Although the growth of product sales in Kazakhstan by 30% may be a fact driven by low sales in the prior period, for example, it doesn’t change the overall signal that worldwide sales continue declining by 4%. Managing information requires business understanding and is less dependent on data management skills.

Insight

With insight, the analysis is moving from what has happened in the past to why did it happen and what does it mean for the future. This is probably the most complex part of the data analytics value chain. It requires a deep understanding of the business and market context to explain why the company has lost market share in the consumer segment last quarter, for example. What were the drivers of that loss? Was it due to internal company decisions, was due to competitor moves, was it due to economy conditions, or was it due to other market forces previously not accounted for? Even more complex is the question what does this all mean for the future. Here you not only need the business acumen and deep domain knowledge, you also need the advanced quantitative skill set to model the future. Expertise with regression, simulation, optimization, and/or diffusion models are critical to arrive at any reliable projection. This predictive analytics part of the value chain requires a broad skill set, but it can only be successful once the prior stages, data and information, have been completed.

Business decisions

Once the history has been analyzed and there is a sufficient understanding about what it means for the future, it’s time to maximize shareholder value based on the new insight. These can be operational decisions impacting the next few months or quarters, or strategic choices impacting the next few years. Business and market knowledge are key in this step, but this stage also requires understanding of the analytical approaches used to arrive at the insight. Although some organizations separate the business decision function from the prior three stages of data analytics, they need to be closely linked together. Business investment choices without the knowledge about how the data was clustered, without the understanding what has happened in the past and why, and without taking into account the inherent limitations of modeling techniques will be misleading at best and detrimental to the future business success at worse.

The above descriptions of the data analytics value chain identified the type of activities that need to be completed during the four stages and an overview of the skill sets required. There are no standardized job titles that correspond with these activities. While one organization may use the title “Business Intelligence Manager” to define only the data analysis role, another will use the same title to describe the function managing the entire data analytics value chain. These disconnects won’t be solved any time soon, so it’s critical for executives, hiring managers, employees, and as well as job applicants to clearly define the expectations not based on job titles, but based on the actual work that needs to be completed throughout the data analytics value chain.