The State of the Data Catalog and Active Metadata Management Market

Organizations are increasingly relying on data management solutions to automate and optimize their operations. One crucial component of these solutions is a robust data catalog that provides comprehensive metadata management capabilities.

The coverage for this space has been evolving over the last few years. Interestingly Gartner covers the area under Metadata Management , and Forrester previously had a Wave that lists vendors under the Machine Learnings Catalog category , with Alation listed as the leader - notably though that Wave has not been updated since 2020.

Ironically in 2018, when I was the CMO and CPO at Reltio , when this space was being formed, we were considered a leader in this space by Forrester

Forrester also has an updated Wave published in 2022 called "Enterprise Data Catalog for DataOps " - with Atlan , who's Market presence is relatively small compared to the other players, clearly listed as a leader.

Another good resource is Garner's Peer Insights for Enterprise Metadata Management in which a full listing of vendors, are rated by actual users with real world deployment and usage experience.

The list includes

• Alation Data Catalog
• Atlan
• Collibra Data Intelligence Cloud
• data.world
• erwin Data Intelligence
• Informatica Enterprise Data Catalog
• Oracle Enterprise Metadata Management
• Microsoft Azure Data Catalog

and many more. So certainly there's been a lot of shifts in the market. So you can't blame potential customers from being confused.

With that all in mind. Let's delve into the active metadata market which encompasses a broad spectrum of data management tools that enable significant active metadata use within their platforms. Active metadata management involves continuous analysis of users, data management, systems/infrastructure, and data governance experiences to align data as designed with actual experiences. It incorporates operationalizing analytic outputs through operational alerts and generated recommendations, which drive AI-assisted reconfiguration of data and active metadata utilization. In 2021 Prukalpa, co-founder of Atlan wrote the visionary medium post "What Is Active Metadata, and Why Does It Matter?" in which she describes the difference between Passive and Active Metadata

Essential capabilities in the active metadata management market include ML over profiling, content analysis, user/use-case clustering, resource allocation metrics, alerts and recommendations, ML by case example with trend and usage, orchestrate recommendation and response, and use case to new asset inference. The market is continues to shift with a focus on market adoption and the business value of metadata sharing. Passive metadata continues to drive growth, but active metadata management is gaining traction. Established metadata management solutions are leveraging active metadata practices and techniques, while new players are entering the market with branded active metadata management solutions.

Interest in active metadata has seen significant growth, highlighting the increasing demand for comprehensive metadata management capabilities. Purpose-built metadata management tools are facing challenges from adjacent data management platforms, such as databases, data integration, data quality, and data governance tools. Many companies like Alation, Collibra and the mega vendors like IBM, Informatica do offer multiple products for each of these categories. Most of them acquired, and pulled together into a suite or offered individually.

As Gartner has noted, organizations are seeking active metadata?to achieve augmented data management capabilities, hence the trend towards “active metadata” that enables continuous access and processing of metadata. This ensures ongoing analysis, design recommendations, and operational alerts for improved decision-making.

Existing metadata management tools fall short?in that they are increasingly incapable of fulfilling comprehensive metadata needs in the enterprise. As a result, organizations are exploring advanced metadata functionality from mature metadata solutions or embedding metadata capabilities within other data management platforms.

Lack of common metadata standards?poses a significant challenge for metadata sharing and interoperability across multiple metadata management solutions in the market. This further emphasizes the need for flexible and adaptable metadata management solutions.

So what are five things a company considering data catalogs and metadata management can do today?

1. Adopt prescriptive capabilities offered by certain metadata management solutions, with parameterized recommendations to alter design inputs. These solutions should exploit adjacent data management tools, such as data observability tools, for effective operations.
2. Share internal metadata?via data management tools and platforms that are able to support broader platform-to-platform orchestration and enhance interoperability across the data management ecosystem.
3. Capture runtime metadata?including data usage, data affinity, and user behaviors. By automating metadata capture, organizations can unlock the value of metadata and drive automation in their data management processes.
4. Import and export metadata?to facilitate metadata integration, processing instructions, and optimization strategies to enhance metadata-driven decision-making.
5. Enable automated system changes?by leveraging metadata analytics workflow management capabilities in adjacent data management systems. Collaborative design capabilities can be the key to enabling seamless integration and efficient metadata-driven operations.