What are the common causes and consequences of poor data quality in analytics projects?

1 Human errors

One of the main sources of poor data quality is human error, which can occur at any stage of the data lifecycle, from collection, to processing, to analysis, to reporting. Human errors can include typos, mislabeling, duplication, omission, or incorrect interpretation of data. For example, a survey respondent may enter a wrong or invalid answer, a data entry clerk may mistype a value, or an analyst may use a wrong formula or filter. To reduce human errors, it is important to implement data quality checks, validations, and audits, as well as to train and educate data users and stakeholders.

2 System issues

Another common cause of poor data quality is system issues, which can affect the performance, reliability, or compatibility of the data systems and platforms. System issues can include hardware failures, software bugs, network disruptions, security breaches, or integration problems. For example, a server may crash, a program may have a glitch, a connection may be lost, or a data format may not be compatible with another system. To prevent or resolve system issues, it is essential to have a robust data infrastructure, backup and recovery plans, and regular maintenance and updates.

4 Business impacts

Poor data quality can have a detrimental effect on the business objectives of analytics projects, such as decision making, customer satisfaction, operational efficiency, innovation, and competitiveness. For instance, poor data quality can lead to poor decisions based on inaccurate, incomplete, or outdated data, resulting in missed opportunities, wrong actions, or costly mistakes. Additionally, it can cause poor customer satisfaction due to unreliable, inconsistent, or irrelevant data, which can affect the quality of products, services, or interactions, and damage the reputation or trust of the business. Furthermore, it can cause poor operational efficiency due to wasted time, resources, or efforts, spent on collecting, cleaning, or correcting data, or dealing with data issues or errors, which can reduce the productivity or profitability of the business. Lastly, it can lead to poor innovation due to limited, biased, or misleading data, hindering the creativity, experimentation, or discovery of new insights, solutions, or opportunities.

5 Best practices

In order to reduce the risks and effects of inadequate data quality in analytics projects, it is recommended to adhere to some best practices. This includes defining data quality standards and metrics based on the business needs, goals, and expectations, and measuring and tracking the data quality performance and improvement over time. Additionally, data cleansing, validation, transformation, integration, and enrichment should be implemented using automated or manual methods, or a combination of both, depending on the data characteristics and requirements. Furthermore, data owners, stewards, custodians, and users should be assigned the authority, accountability, and resources to ensure and improve the data quality. Finally, a data quality culture and mindset should be fostered among the data users and stakeholders, and the awareness, education, communication, and collaboration on data quality issues and solutions should be promoted.