Data science is easy. Change is hard.

After thirty years of professional experience, I continue learning the same lesson: technology is the easiest part of my job while helping people change is the hardest part of my job. Although creating data science solutions is complex, compelling people to use data-driven predictions is very complex. Success of data science projects is defined by business improvements, not by creating models. A business improvement is achieved when people change how they work. Because approaches to data science projects focus on data rather than people, the business may not use solutions created by data science teams. People change is a messy, feeling-driven process that calls for the use of non-technical techniques. If you include people change tasks in your data science projects early on, your chances for success will rise.

Success is an improvement achieved by people

How do you define a successful data science project? Is success based on how well a model predicts an outcome? Expected value? Lift? Some data science teams believe a project ends when they create a model that meets a set of pre-defined requirements. I argue that a data science project is successful when a business improvement is recognized. Saving money, improving reliability, reducing inventory, decreasing risk, and increasing sales are examples of improvements. An improvement might be achieved by solving a one-time problem or the improvement might be achieved by enhancing a repeatable process. In any case, an improvement is achieved when someone acts based on a data science solution, resulting in better conditions. Whether the solution is a model, report, dashboard, or interactive software, the solution itself does not generate an improvement, a person does.

For a person to generate an improvement using a data science solution, change is required. In my experience, most data science solutions aim to improve an existing decision. Usually, the business is estimating something and wishes to improve the quality of the estimate. In simple circumstances, the business may start using a new number produced by a data science solution. In complex cases, the business may have to modify work processes carried out by thousands of employees, all of whom must accept and enact the changes. In all cases, simple and complex, business improvement only occurs if people change how they work.

Reflect on the reasons why data science projects fail. One common symptom will emerge: the business did not use the solution. The phrase I hear often is “the business did not adopt the solution.” Reflect on the reasons why data science projects succeed. A different common symptom will emerge: the business used the solution. You probably did not think: the solution did not work. Since data science tools are flexible, easy to use, and embed math and automation, one rarely needs to understand the underlying algorithms that drive the models. Conversely, no tools exist that embed the methods for changing human behavior.

People change is largely absent in data science methods

Some approaches to data science projects minimize time spent on people relative to time spent on technical tasks. In the written domain on data science methods, much time is devoted to data: gathering data, cleaning data, correcting data, aggregating data, splitting data, modeling data, and visualizing data. But data does not do anything, people do. Regardless of how elegant and insightful your data science solution may be, it is worthless if left unused. Some research claims that most data science projects fail. I see why: helping people change seems like an afterthought in data science project methodologies.

As an example, one popular methodology used to implement data science projects is called CRISP-DM (CRoss Industry Standard Process for Data Mining). This methodology appears to be the most referenced approach to guide data science projects. The figure below depicts the steps and workflow of the CRISP-DM methodology:

CRISP-DM has many qualities: it is simple to understand, easy to follow, and includes technical tasks that should be executed during data science projects. However, the methodology is mostly silent on helping people change. The last step in CRISP-DM is Deployment. The description of this step includes the following phrase: “it is important for the customer to understand up front what actions need to be carried out in order to actually make use of the created models.” Ironically, the task shows up at the end of the process, not up front. Further, notice that “Data” is in the center of the process, suggesting data is the key to success. Data is a digital record of information that is incapable of action. Only people can put data into use to make an improvement. The word cloud below was generated from the entire text of CRISP-DM.

In the above word cloud, you do not see words like “people”, “change”, “communication”, or “training.” The concept of people change is not prominent in this methodology, though people change is required for success.

Have you spent months (or years) developing a data science solution that goes unused by the business? If you exhausted time gathering requirements, attending countless meetings, and working long hours modeling data, only to learn that your hard work was put on a shelf, you might feel defeated. Failures can be avoided if you include the following tasks up front in your data science projects.

Define the improvement

A data science project is successful when an improvement is recognized. Before an improvement can be recognized, the improvement must be defined. An improvement should be discrete and not ambiguous. “Make data-driven decisions” is an ambiguous improvement. “Reduce spending by 10%” is a discrete improvement. If the definition of the improvement is not evident, you may need to guide a conversation with the business to develop the definition.

My favorite approach to developing a definition of a discrete improvement is to ask, “So what?” You may have heard of asking “Five whys” to help discover one or more root causes of a problem. Asking “So what?” helps discover one or more benefits of solving a problem. Here is an illustrative example of an exchange with the business using the question “So what?” to uncover an improvement:

From here, you can drill deeper: “How many widgets are we buying now?”, “What are our current widget inventory levels?”, “For every 1% reduction in widget inventory levels, how would cash flow and profits be improved?” Keep the conversation going until you have fully described, as discretely and quantitatively as possible, the desired improvement.

The business may not have answers to these questions before calling the data science team for help. The business may feel frustrated being interrogated because they have a business problem that needs to be fixed, so engaging in the above conversation may feel like a waste of time. However, if you remain respectful and curious, the business will usually appreciate this exercise because you are expressing a deep interest in understanding their problem and you are helping them better define their problem.

Defining the improvement should take no more than one day. Bring together the right people from the data science team and the right people from the business team to define the improvement. Something else may happen when you bring people together to define the improvement: people may start feeling excited. People may begin to picture how their life might be made easier with the improvement in place. The exercise also helps the business think about changes that may be required to realize the improvement.

Generating excitement in the business is important because the business must eventually act on the data science solution if they hope to realize an improvement. If the business feels excited about the improvement, chances are, when the time comes, they will use the data science solution to act. Spending time discussing an improvement brings team members closer together and galvanizes the teams’ understanding of what success looks like.

Stop if you cannot define the improvement. You may not have the authority to stop a data science project. However, if the improvement cannot be defined, you should try and stop work until you can. Even if you have volumes of high-quality data and a crackerjack team of data scientists, you cannot claim success if you do not know what success looks like.

Define the change

Once the improvement is defined, discover how people must change for the improvement to become a reality. Before the data science solution is defined, the data gathered, and the models trained, one can envision change. Defining change early on requires imagination. In the case of a data science solution, defining change involves imagining a future in which a new tool improves decision-making. Unfortunately, defining the change is harder than defining the improvement.

“If you had the answer, what would change?” Asking this question is the simplest way to tease out a definition of change with the business. Most data science projects involve improving an existing decision process. For example, the business may be manually estimating a number and wants the data science team to provide a better estimate or perhaps provide an exact recommendation. In this example, because the business follows an existing process, it is easier to imagine how the process might change if a better estimate is provided. The table below provides an illustrative example of an exchange with the business to help define the change.

By defining changes early, albeit at a high level, the business and data science teams begin to recognize that if changes are not made, the improvement will never be recognized, spelling failure for the data science project.

I can hear my data science colleagues arguing: “But what about the data? What if we don’t have good data? The project will fail if we don’t have data!” Not having enough quality data may spell failure for a data science project. Not changing how people work is guaranteed to spell failure. Lacking data and failing to change are both serious risks faced by all data science projects. Mitigate the change risk first because the effort to define change is far less than the effort to discover if there is enough good data. In one day, you will know whether the business can envision the change they must undertake. How many days does it take to figure out if you have enough good data?

Define a change plan

Once the change has been defined, brainstorm the specific tasks required to affect the change. Define a change plan working with the same business stakeholders who helped you define the improvement and the change. Because most data science projects improve the outcomes of an existing decision process, the change plan should reflect actions that help people transition from the old decision process to the new decision process. Most tasks involve personal communications, interviews, documentation, and training. The changes identified in the example above have been copied into the table below with tasks added to provide an illustrative example of a change plan:

Next, for each task, identify the leader, the completion date, and the dependencies on other tasks. This example plan is very simple, involving few people working on a small-scale data science project. Your data science project may involve many people and require a larger variety of change tasks in your change plan. Although the effort to build the change plan may be significant, the plan lays out the path for the successful adoption of the solution by the business.

“What about the data?” When you feel comfortable that the business understands the improvement and accepts the types of changes that must occur to be successful, begin your data tasks. Because each data science project is unique, the exact timing of starting data tasks will vary. Generally, I suggest beginning your data tasks in parallel with defining the change plan.

Implement the change plan

If you have an existing plan for your data science project, include the change tasks in your plan. Track the progress of change tasks the same way you track technical data science tasks. If you follow an agile approach to development, include these tasks in your sprint plans, discuss change blockers in your stand-ups, and enlist the business to help implement tasks in your change plan. If you have a project steering committee or a project advisory panel, discuss the change plan and solicit their help to resolve change issues. If the change plan cannot be successfully executed, your data science project will not be successful. ?

Conclusion

I love data science. Discovering hidden patterns in large volumes of data using advanced analytics techniques is thrilling. Outside of academia, the reason for investing in data science teams is to improve business outcomes, not to create models. I wish data-driven discoveries would automatically improve business outcomes. In reality, models produced by data science teams are worthless unless put into use by the business to make improvements. To be successful at data science, take on the challenge of helping the business change. Otherwise, a lot of hard work may be for naught.

Andy Quick has thirty-two years of professional experience in technology and business. He currently leads an enterprise data analytics team.