On to Data Quality

My last article (Laptop Data Quality) included two questions, both related with a perspective that Data Quality is everybody’s responsibility, hence it’s no one’s responsibility and the fact that it’s common for organizations to informally delegate this responsibility to data users, diverting them from their work to handle the quality of the data they need.

Those two questions were:

1. How many hours are being spent across the organization in ad-hoc tasks related with data quality, and what is the cost of those hours?
2. What is the effective cost, or value not being generated, due to the hours that are diverted to these tasks?

?None of those has an easy answer because both the costs and benefits remain hidden in the original work of the analyst, developer, data scientist or any other data user.

This not only impacts in their performance it can also impact the results as it depends mostly on their context, needs, and know how – and the same data will be corrected differently by different people.

A more structured approach to data quality might not be easier, typically these are expensive initiatives; they are time and resource consuming and span through long time frames.

They can also be deeply intrusive and disruptive, creating the natural resistance to change within the organization, creating an incredibly challenging ecosystem to work on.

Finally, we are talking of the kind of initiative that might take years to break even and deliver ROI, making it hard, even with a strong sponsorship, to keep the necessary traction to complete all the necessary changes.

These are the most frequent causes I have seen identified on “project post-mortem” reports, and it’s easy to understand why these projects are rarely a priority. No one wants to own a project where it’s not easy to evaluate the impacts, implies a lot of effort and will only show return on the long term.

Impact of Bad Data

The impacts of having bad data in the business processes are easy to identify and affect every single business are and process, cutting across the entire organization.

These impacts may be in the form of:

1. Lost revenue, sales, or business opportunities, including:

• Lost sales opportunities.
• Failure to do product cross-selling.
• Impairment in properly identifying customer’s needs.
• Failed marketing campaigns.
• Invoicing problems, either resulting in an inability to properly bill the customers or in additional costs in the billing process.
• Missed B2B opportunities or inefficient procurement due to the incapability to accurately analyse the market.

?2. Customer dissatisfaction and service costs, including:

• Loss of a dissatisfied customer, that besides the direct cost related with the customer lifetime value added to the costs associated with new customer acquisition, can also imply indirect costs, as the customer can work as a market influencer, leading to the loss of prospects.

3. Operational inefficiencies, including:

• Poor resource planning.
• Increased operational costs, either on system workloads or work hours spent on data quality related issues.

4. Regulatory compliance, including:

• Inability to comply to regulatory compliance. In some industries where regulatory compliance is essential, poor data quality has a significant impact on the capability to comply with the regulatory obligations, resulting in heavy monetary penalties or even civil or criminal proceedings.

5. Poor decision making, including:

• Inability to make correct long-term decisions.
• Incorrect forecasts.
• Inaccurate customer profiling and segmentation, leading to decreased sales and customer retention.

How to approach

To avoid the traps of entering on a Laptop Data Quality mode, or the ones of a major Data Quality Program we should try to address specific problems with known consequences, so we can define Data Quality Initiatives that:

• Have reasonable funding model
• Are targeted
• Have focused effort
• Have short timeframes
• Increase internal engagement
• Deliver targeted return on a short timeframe

In large organizations, even the ones without a strong data culture, the opportunities to start these initiatives are quite abundant. Across all the business areas there are pain points related with the quality of data and identifying them is not a challenge.

We just need to find that business stakeholder that can passionately and effectively articulate the impacts of poor data quality in their processes and help him solve the source of his problems.

Minding that most of the time it is not about identifying the actions that can reach the best ROI but identifying who is the one that has a problem that needs to be targeted, assessed, and mitigated quickly.

It is easier to help someone that asks for help than persuading someone that it needs help.

A sequence of these targeted initiatives has the benign effect of increasing the awareness of the importance and impact of data quality across the organization, increasing the overall internal engagement, turning critics into evangelists, and paving the way to a more structured and strategic approach enterprise wide.

On to Data Quality

A successful data quality program may be a daunting task, although not impossible depending on the approach.

Focusing on the characteristics mentioned above one of the possible approaches is to follow these steps:

1. Start with business areas than can clearly identify and measure the business impact of bad data on their processes

In every organization the opportunities to identify these cases are abundant. Across all the business areas there are pain points related with the quality of data and identifying them is not a challenge.

2. Build your business case with those willing to defend it

Once you have identified a critical pain point, you will have the business stakeholder that can passionately and effectively articulate the impacts of poor data quality in his processes and that will be eager to defend the project.

3. Focus on turning insights into action

Having the business stakeholder working by your side will accelerate the process of quickly move from the findings to specific actions.

4. Establish data quality targets based on critical data for business

A deep understanding of the impacts of bad quality data on the business processes enables a more accurate prioritization of the critical data, hence making easier to identify clear targets on an early stage.

Making the option for data quality initiatives that are more focused and efficient creates and increases the awareness across the enterprise and ends acting as the motor from within the organization for a full Data Quality Program.