You're facing conflicting data sources for your report. How do you navigate this challenge effectively?

If multiple data sources are providing similar data, a policy should be set to follow a specific data source as the official data. As long as the data is consistent with accuracy and measurable guidelines, you can provide valid insights and direction. We make it a priority to eliminate any reporting that doesn't come from the official data source but still retain that data for future reference and analysis in case it is needed as the business changes direction.

Assess the credibility of each data source: Look at factors like the author's qualifications, the methodology used, and the reputation of the source. Prioritize more credible and authoritative sources. Understand the data collection and processing methods: Try to get details on how each team collected and processed the data. Identify any differences in their approaches that could lead to the conflicting results. Reconcile the differences through further analysis: Dig deeper into the data to understand the root causes of the discrepancies. Look for patterns, outliers, or other insights that could help explain the conflicts.

Analyse all sources, and meet the business and/or data owners. create a golden record model. then and only then if you still have conflict, establish ownership rules.

Although you don't typically want conflicting data sources, analyzing the conflicts can lead to some very productive conversations. Discrepancies can be caused by one team looking at a different snapshot of the data compared to another team or one team has access to a much higher level of data granularity leading to differences post-processing. For me, uncovering the reason for such discrepancies has not only brought more voices into the data governance and management conversation, but has also enabled cost savings in terms of data storage and processing.

This is a common issue if you have datasets from different systems. E.g. 'mailAddress' can have different values in the invoice system and sales/delivery system. Both are correct, but one may be an office on the 15th floor, and the second, a warehouse. Make sure you are comparing apples to apples. Another potential reason, maybe the input form in system A makes a field Z mandatory, while the data is irrelevant for business, so all users learned to type '11' just to remove the alert on submit. Learn user behaviours, understand their stories

This is a common issue if you have datasets from different systems. E.g. 'mailAddress' can have different values in the invoice system and sales/delivery system. Both are correct, but one may be an office on the 15th floor, and the second, a warehouse. Make sure you are comparing apples to apples.

This is an extremely important step, but usually only documented in emails or meeting notes. Every additional data modification done after user/system input must be documented in a business analysis document (as a reason) and technical design (as a method). Otherwise, in five years during system migration, nobody will remember what was the reason for this weird regexp.

Faced with conflicting data sources for a crucial report, a data manager felt the weight of the looming deadline. * To navigate this challenge, they first identified and verified the primary sources of each dataset. * Engaging with key stakeholders, they discussed discrepancies and sought clarity on the data's context and collection methods. * They employed statistical techniques to reconcile differences and documented the reasoning behind their decision. By fostering transparent communication and leveraging analytical rigor, they delivered a comprehensive, accurate report, reinforcing trust and showcasing their problem-solving prowess.