Choosing Data Management IT Tools: Metadata Management Solutions

The previous article ?reviewed specifics in selecting data governance solutions and their functionalities. In this article, we will discuss the following:

• The definition and content of metadata management and related challenges
• Business needs and requirements for a metadata management tool
• The situation with commercial-off-the-shelf (COTS) data governance tools (based on the analysis of 40 tools)

Metadata Management: Definition, Content, and Associated Challenges

Metadata

Let’s start with the definition of metadata. In my practice, I use the following definition: “Metadata is data that defines and describes other data in a particular context.”

Metadata can be of various types. Multiple metadata classifications exist. DAMA-DMBOK2 recognizes three: business, technical, and operational, as shown in Figure 1.

The definitions of these metadata types are presented in Figure 1.

Several challenges are associated with metadata:

1. Metadata can be data in a particular context.

Let’s take data instances in a database column. These data instances don’t have meaning unless we provide a technical name to this column. The technical name is metadata. We may also need to explain the meaning of the technical names. It will require metadata at an upper level. In other words, we need metadata for metadata. In this case, the technical names turn out to be data in a particular context.

1. Metadata can be a single data element or a complex object.

An application owner is a single data element. A data model is a complex metadata object that consists of multiple elements.

1. Various metadata types represent the same object.

Data lineage in the viewpoint of DAMA-DMBOK2 combines business and technical metadata.

Metadata model

Figure 2 demonstrates an example of a metadata model I developed for data lineage. A metadata model shows the metadata objects and the relationships between them. Data lineage is a complex metadata construct that combines business and technical metadata.

I described this metamodel in my book, “Data Lineage from a Business Perspective .”

The key message is the following: various metadata objects and elements must be taken into the scope of metadata management. They are not limited only to the physical level. These objects and elements can be found in multiple IT applications.

Metadata management

Let’s start with the definition of metadata management provided by?DAMA-DMBOK2 : metadata management is “Planning, implementation, and control activities to enable access to high quality, integrated metadata.”

I apply another definition in my practice: “Metadata management is a company’s ability to discover, gather, and integrate metadata of required quality to enable a data lifecycle.”

Several challenges are associated with metadata management.

1. Many companies don’t pay enough attention to establishing metadata management.

Metadata management is an enabler of a data lifecycle, including data integration. Metadata management focuses on gathering, integrating, and distributing metadata. In other words, metadata management enables data and metadata lifecycles. That is why proper data management is only possible with metadata management.

1. Many professionals limit metadata management to technical metadata.

Metadata is a product of various capabilities and corresponding IT tools like business process management, data modeling, data-, application-, technology architecture, data quality, and multiple IT infrastructure-related capabilities. Different types of metadata must be gathered and integrated. Knowledge graphs are one of the examples of this requirement.

1. Metadata volumes can exceed data volumes.

Data consumption and production volumes grow exponentially. It causes an increase in metadata volumes. In my opinion, the metadata volumes exceed data volumes.

Metadata management requires the same capabilities as data management, as shown in Figure 3. The simple reason for that is that metadata is also data. Figure 3 presents the capability model of metadata management.

These capabilities include metadata governance, quality, modeling, and architecture. You can find more information in the online course: “Designing metadata management and data lineage capabilities .”

The above-discussed challenges with metadata and metadata management strongly influence requirements for metadata management tools.

Business needs and requirements for metadata management tools

Often, professionals ask me the same question: “We decided to implement metadata management or data lineage. What kind of tool do we need?” My answer starts with the counter-question: “What are your requirements?”

The role and use cases of metadata management are the best examples of a company’s needs in metadata management:

• Enabling data integration

Companies tend to integrate data from multiple internal and external sources to get more insight from data and support business decisions. Metadata describes data and enables its integration.

• Improving efficiency by identifying duplicated and redundant data

Large companies have hundreds and thousands of various applications. Data is often duplicated. Much data is not being used for a long time. Operational metadata can assist in identifying these issues.

• Establishing traceability and transparency of data processing, transformation, and integration because of regulation requirements

To comply with various regulations, organizations must document data lineage which combines business and technical metadata.

• Reducing IT and DevOps costs

Properly organized metadata reduces time and effort in developing new applications and optimizing data & application landscapes.

Modern metadata management introduces several concepts or capabilities that assist in meeting the business needs discussed above.

These capabilities include data lineage, knowledge graphs, observability, and active metadata.

Read further: https://datacrossroads.nl/2023/08/16/choosing-data-management-it-tools-metadata-management-solutions/

?

?

About the author:

Dr. Irina Steenbeek is a well-known expert in implementing Data Management (DM) Frameworks and Data Lineage and assessing DM maturity. Her 12 years of data management experience have led her to develop the "Orange" Data Management Framework, which several large international companies successfully implemented.?

Irina is a celebrated international speaker and author of several books, multiple white papers, and blogs. She has shared her approach and implementation experience by publishing?The "Orange" Data Management Framework,?The Data Management Toolkit,?The Data Management Cookbook, and Data Lineage from a Business Perspective.

Irina is also the founder of Data Crossroads, a coaching, training, and consulting services enterprise in data management.?

To inquire about Irina's training, coaching, or participating in your company webinar or event, please, email to?[email protected] ?or book a free 30-min session at https://datacrossroads.nl/free-strategy-session/