登录查看更多内容

How can you enrich data with metadata throughout its lifecycle?

由人工智能和领英社区提供技术支持

Data is a valuable asset that can be used for various purposes, such as analysis, decision making, innovation, and compliance. However, data alone is not enough to achieve these goals. You also need metadata, which is the information that describes, defines, and documents the data. Metadata can help you understand the context, quality, and lineage of your data, as well as enhance its usability, accessibility, and security. In this article, you will learn how to enrich your data with metadata throughout its lifecycle, from creation to deletion.

此文章中的业界达人

由社区从 5 条内容中精选。了解更多

Viswa Subramanian Sekar

EY | Data & AI Strategist | DE Chief Data Office | Technology Consulting
Jim Barker
Soon Heng TAN ???? ???? ??????????

Head of Data & Analytics (Data Management) at Public Service | Master of Technology (Knowledge Engineering) | CDMP |…

1 Identify metadata sources

When it comes to enriching your data with metadata, the first step is to identify the sources. Metadata sources can be divided into three categories: data sources, data processes, and data users. Data sources provide metadata such as format, schema, location, and frequency; whereas data processes provide metadata such as lineage, provenance, quality, and rules. Lastly, data users provide metadata such as definitions, business terms, policies, and feedback. To ensure you capture all the relevant metadata from these sources, you should inventory and catalog them and define how to store and update the metadata they provide.

添加您的观点

Soon Heng TAN ???? ???? ??????????

Head of Data & Analytics (Data Management) at Public Service | Master of Technology (Knowledge Engineering) | CDMP | TOGAF | AWS-SAA | GCP-DE | GCP-MLE | DCS-DS | CSM | CSSLP | EDHL Enterprise Data Harmonisation Leader??
举报内容
Manage metadata updates according to business priorities can be achieved through a structured approach: 1. Identify Key Metadata Categories - Start by defining essential metadata categories that align with business objectives. These include technical metadata (like data source, format, and schema), business metadata (such as purpose, owner, and related business processes), and operational metadata (like access controls, usage statistics, and quality indicators). 2. Prioritize Based on Business Impact - Understand which metadata elements have the most significant impact on business operations, decision-making, or regulatory compliance. Focus on enriching, updating metadata that directly supports critical use cases or strategic initiatives.

已翻译

赞
Jim Barker
举报内容
When building out the inventory of governed objects in a catalog for a data governance program it is critical to start with what sources (system) and tables are needed to be governed. You never want to govern everything and spending a bit of time to determine the sources that will have benefits will limit the work, increase success, and drive positive outcomes.

已翻译

赞

2 Implement metadata standards

The next step to enrich your data with metadata is to implement metadata standards that ensure consistency, accuracy, and interoperability. Metadata standards are rules or guidelines that specify how to name, format, structure, and classify your metadata. Examples include data naming standards for avoiding ambiguity, duplication, or confusion; data modeling standards for defining the logical and physical structure of data; and data classification standards for categorizing data based on its characteristics. It is important to adopt and enforce metadata standards across your data sources, processes, and users. Additionally, you should use tools or methods to validate and monitor your metadata compliance.

添加您的观点

Jim Barker
举报内容
One key about metadata standards is to determine defines a fully curated definition for a data object. It also includes being able to define what fields are critical data elements, and what you need to document for these critical fields.

已翻译

赞

3 Integrate metadata systems

The third step to enrich your data with metadata is to integrate your metadata systems that store and manage it. Metadata systems can be classified into two categories: technical metadata systems, which store and manage the technical or low-level metadata, such as data format, schema, location, or lineage; and business metadata systems, which store and manage the business or high-level metadata, such as data definitions, business terms, policies, or feedback. To enable the exchange and synchronization of your metadata across different levels, domains, and functions, you should use methods or technologies such as metadata extraction, ingestion, mapping, or federation. These systems can be embedded in your data sources or processes or can be standalone or integrated with them.

添加您的观点

Jim Barker
举报内容
As data platforms have their own metadata repositories the need to integrate catalogs together to have a catalog of catalogs is very important. Understanding the overall landscape of metadata repositories and desired outcomes in important to both understand and document.

已翻译

赞

4 Share metadata insights

The fourth step to enrich your data with metadata is to share your metadata insights that reveal the value and meaning of your data. By doing so, you can answer questions such as what the data is, where it is located, how it is, why it exists, and how it got there. Metadata insights such as data names, types, domains, categories, locations, systems, owners, quality scores, rules, issues, policies, standards, feedback, transformations, dependencies and changes can help you answer these questions. It is important to share your metadata insights with data users and stakeholders and use tools or methods to visualize, communicate and collaborate on your metadata.

添加您的观点

Viswa Subramanian Sekar

EY | Data & AI Strategist | DE Chief Data Office | Technology Consulting
举报内容
Sharing metadata insights is like storytelling - right story, to the right people, at the right time. Metrics are key and it's not just about tech lingo, business metrics equally important. Craft various dashboards for different audiences. E.g.: For data office team, showcase stats on onboarded business units, tackled use cases, onboarded critical data sources etc. Business dashboards could showcase no. of well-documented critical assets, glossary terms, % owners, stewards etc. Tech dashboards could track quality metrics, stewardship status, error counts etc. Beyond metrics, it's also about cultivating a culture - Amplify success stories, conduct regular community exchange calls and incentivize good data management practices if possible.

已翻译

赞

5 Update metadata continuously

The fifth step to enrich your data with metadata is to update it continuously, to reflect changes and evolution of the data. Data is dynamic, so your metadata should be kept up-to-date in order to maintain accuracy, completeness, and timeliness. You can update your metadata in two ways: manually or automatically. Manual update involves human intervention, such as editing, reviewing, or approving, and is recommended for subjective, complex, or sensitive metadata like data definitions, business terms, or policies. Automated update requires system intervention such as scanning, parsing, or validating and is suitable for objective, simple, or frequent metadata like data format, schema, or quality. To ensure your metadata is updated properly and consistently, you should use tools or methods to track, audit, and notify changes.

添加您的观点

6 Delete metadata responsibly

The sixth and final step to enrich your data with metadata is to delete your metadata responsibly when your data is no longer needed or useful. Data deletion is the process of removing or destroying your data, either permanently or temporarily, due to reasons such as data retention, obsolescence, or compliance. You should delete your metadata in sync with your data, to ensure that your metadata reflects the current state and availability of your data. Additionally, you should delete your metadata across all your metadata systems, with backup and permission. To avoid metadata fragmentation or duplication, you should delete your metadata backup according to your data backup policy. Furthermore, you should delete your metadata with notification, to inform your data users and stakeholders about the deletion. You should use tools or methods to document, verify, and report your metadata deletion.

添加您的观点

7 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?

添加您的观点

Data Governance

+ 关注

给文章评分

我们借助人工智能创建了此文章。您认为这篇文章怎么样？

很棒不太好

举报此文章

查看全部

How can you enrich data with metadata throughout its lifecycle?

1

2

3

4

5

6

7

1 Identify metadata sources

2 Implement metadata standards

3 Integrate metadata systems

4 Share metadata insights

5 Update metadata continuously

6 Delete metadata responsibly

7 Here’s what else to consider

Data Governance

给文章评分

感谢您的反馈

更多Data Governance相关文章

更多相关阅读内容

How can you enrich data with metadata throughout its lifecycle?

1

2

3

4

5

6

7

1 Identify metadata sources

2 Implement metadata standards

3 Integrate metadata systems

4 Share metadata insights

5 Update metadata continuously

6 Delete metadata responsibly

7 Here’s what else to consider

Data Governance

给文章评分

感谢您的反馈

查看其他技能