How do you harmonize metadata from different domains?

1 Identify the metadata sources and scope

The first step is to identify the sources and scope of the metadata you need to harmonize. You may have different types of metadata, such as structural, descriptive, administrative, or provenance metadata, depending on the nature and purpose of your data. You also need to determine the level of granularity and detail of the metadata, such as whether you need to harmonize metadata at the dataset, table, column, or value level. You should also consider the frequency and mode of metadata updates, such as whether they are manual, automated, or event-driven.

2 Map the metadata elements and relationships

The next step is to map the metadata elements and relationships across the different domains. You need to identify the common and unique metadata elements, such as attributes, values, and labels, and how they relate to each other. For example, you may need to map the metadata elements of a customer ID in the finance domain to the metadata elements of a user ID in the marketing domain. You also need to define the rules and logic for mapping the metadata elements, such as whether you use exact matches, partial matches, synonyms, or transformations.

3 Choose the metadata harmonization approach

The third step is to choose the metadata harmonization approach that best suits your needs and constraints. Standardization involves adopting a common metadata standard or schema across different domains, such as ISO 19115 for geospatial data or Dublin Core for web resources. This requires a high level of coordination and agreement, but it can simplify metadata management and integration. Mediation uses a middleware or mediator system to translate and reconcile metadata from different domains without changing the original sources. This requires a low level of coordination, but it can introduce complexity and latency in processing. Federation creates a federated or distributed metadata repository that links and accesses metadata from different domains without storing or duplicating them. This requires a moderate level of coordination, but it can enable a flexible and scalable metadata architecture.

5 Monitor and evaluate the metadata harmonization outcomes

The final step is to monitor and evaluate the metadata harmonization outcomes and impacts. You need to measure and track the metadata harmonization performance, such as the accuracy, completeness, timeliness, and usability of the metadata. You also need to assess and communicate the metadata harmonization benefits, such as the improved data integration, interoperability, and analysis. You should also collect and incorporate the feedback and suggestions from the metadata users and stakeholders, such as the data engineers, analysts, or consumers, to continuously improve the metadata harmonization process and results.

6 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?