What are the best practices for modeling data lineage and impact analysis in Data Governance?

1 Data lineage models

Data lineage models are visual representations of the data flows and dependencies in your data ecosystem, ranging from the sources to the targets, and from the business to the technical level. They can be categorized into three types: horizontal, vertical, and hybrid. Horizontal data lineage models display end-to-end data movement across various systems and platforms, such as databases, ETL processes, data warehouses, and BI tools. Vertical data lineage models illustrate the data transformation and derivation within a single system or platform, such as the logic, rules, and calculations applied to the data. Hybrid data lineage models combine both horizontal and vertical perspectives, showing the data flows and transformations at different levels of granularity and abstraction. To create effective data lineage models, it is best practice to use a data governance tool that can extract, map, and visualize data lineage from various sources and formats. Additionally, you should define a common terminology and notation for your models (e.g. DAMA Data Lineage Framework), document them with relevant metadata (e.g. data owners, stewards), and validate/update them regularly through manual/automated checks.

2 Impact analysis models

Impact analysis models are analytical representations of data lineage models that help evaluate the impact of any changes or issues on data assets and their downstream consumers and processes. These models can be performed in two directions: forward and backward. Forward impact analysis models display the effect of a change or issue in a data source or transformation on data targets and outputs, such as reports, dashboards, and analytics. Backward impact analysis models demonstrate the impact of a change or issue in a data target or output on data sources and transformations, such as data quality, validity, and compliance. To create effective impact analysis models, it's best to use a data governance tool that leverages the data lineage models and metadata to perform scenarios like what-if analysis, root cause analysis, and dependency analysis. Additionally, you should define and follow a common impact analysis methodology and criteria, such as the Business Impact Analysis (BIA) Framework. It's also important to communicate with your data stakeholders to gather their input and feedback on the impact analysis models and results. Finally, you should monitor and measure the impact analysis models and outcomes using metrics like data quality, reliability, availability, and usability to ensure their effectiveness.

3 Data lineage and impact analysis benefits

Data lineage and impact analysis models can provide many benefits for your data governance initiatives. They can improve the trust and transparency of your data assets by showing their origin, transformation, and usage. Additionally, they can enhance the quality and reliability of your data assets by identifying and resolving any data issues, errors, or inconsistencies in your data flows and transformations. Moreover, these models can ensure the compliance and security of your data assets by tracking and auditing their lineage and impact, as well as applying the appropriate policies and controls to protect them. Furthermore, they can optimize the efficiency and performance of your data assets by streamlining and simplifying your data flows and transformations, while reducing the time and cost of data management and maintenance. Lastly, they can support the innovation and value creation of your data assets by enabling data discovery and exploration, as well as facilitating data-driven decision making and analytics.

4 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?