Practically Improving Data Quality

All organisations have issues with data quality, it’s almost a universal truth. Surprisingly however, not many organisations have continuous activity to improve data quality.

Data quality issues are often addressed only when they're big enough that a significant business impact is experienced. The business then issues a specific one-off initiative to cleanse or correct the bad data. Of course, with the symptom addressed but not the cause, the issue re-occurs and so do others like it across the business.

Here's a few ways to address data quality in a more fundamental way:

Solution & application design

Both the front and backend of your applications should aim to enforce data quality and encourage good data etiquette. It should be easy and quick for users to give you the right data. For example, the drop-down list of countries in your contact information form should have search enabled, both through country name but also country code. In your backend you should be enforcing schemas as data moves around with appropriate actions if the data doesn’t fit the bill.

Measure data quality throughout its lifecycle

With data becoming increasingly accessible, you have the ability to measure data quality without having to build dedicated functionality into existing platforms. When you first extract data into the cloud, plug in your favourite visualisation tool like Power BI or Tableau and use a data quality template to measure the quality. As data moves around the cloud, being used for reporting or integrated into other systems, track its lineage and continue to measure it to ensure that it stays high quality and error free.

Quantify the cost of poor data quality

It can be hard to measure the impact of bad data quality and therefore the problem is only recognised once a critical mass is reached. For example, if 30% of your customer phone numbers are incorrect, how much revenue will be missed during your next marketing campaign? - $10,000, $100,000?

If you don’t know the cost of poor data quality it’s difficult to produce a business case to get investment and generate remediation activity. Therefore, you should aim to build an understanding of the use of data and of the value that’s generated from it.

Implement data governance with a practical focus

Data governance is often simplified down to ownership and stewardship. The common approach is to make sure you have a central data quality forum, that data has business owners and stewards, and then improvements will just happen. However, a combination of technology, people and processes are required to generate impact. Budget, resource, development capability and more are required for data quality remediation. Your central governance should not just be delegating responsibility out to business owners, it should actively identify solutions and support investment for remediation.

Differentiated Data Literacy

Manual data entry is probably the leading cause of poor data quality. Data literacy programmes to address manual entry often focus on blanket training delivered via online courses. These courses are mandatory, uninteresting and easy to ignore. Good professional development should be interactive, practical and collaborative. Therefore, delivery methods, content and more should be customised to meet the needs of each segment of the organisation.

Of course, there are many other methods for improving data quality, some as complex as using machine learning to populate missing data or as simple phoning customers to get missing information. Organisations should begin with the fundamentals to drive results from data governance and management, focusing on driving measurable value.