A Thirst for Knowledge Graphs
In today's data-driven world, information overload is a common challenge, especially for large and mid-sized organizations. Old data tends to get forgotten (read: swept under the Sharepoint), while intense efforts are placed on leveraging new data as soon as it arrives.?
This creates a daunting dichotomy for data management professionals: Should we just forego efforts to understand old data and focus all our attention on what's happening now? Or, should we embark on a journey of digging in the dirt, to quote the amazing musician, Peter Gabriel?
One major force that's driving the second option is the incredible power of Artificial Intelligence (AI), especially the generative kind that's now taking the business world by storm. GenAI promises to revolutionize the creation of meaningful content for all kinds of business needs.
Out of the box, Large Language Models like ChatGPT and Bard can create very compelling prose, providing a great resource to content marketers and related professions. But it is truly generative in nature, meaning it creates new content based upon user prompts.
That new content could be true, or false, which is why many of the major vendors in this space are focused on helping companies train their models to not "hallucinate" - as the practice is called. These hallucinations are more of a feature than a bug, according to OpenAI.?
That's why the next phase of this Generative AI movement is so focused on helping companies train models on their own corporate data. But that brings us to the aforementioned issue of data that was swept under the rug. Do you really know what your corporate data says?
Knowledge Graphs
Some companies actually do! Many of those organizations realized the importance of understanding their corporate data at-scale, and they addressed that need by building a knowledge graph to serve as a foundational component of their information strategy.
Knowledge graphs aren't new, but they are newly popular. Internet titans like Facebook, LinkedIn, Twitter and others use the power of graphs to identify relationships between entities, like individuals. When these social engines recommend connections to you, that's a graph!
A knowledge graph is a data structure that represents knowledge in a graph format, consisting of interconnected nodes (entities) and edges (relationships) that define the connections between those entities.?
It offers a flexible and intuitive way to organize and navigate vast amounts of information. Unlike traditional databases, which store data in rigid tables, knowledge graphs provide a more dynamic and holistic approach to knowledge representation.
The core principle of a knowledge graph lies in the use of semantic relationships, where nodes represent real-world entities, and edges capture the relationships between them. This semantic web of interconnections enables organizations to go beyond simple data storage and retrieval, facilitating complex querying, reasoning, and inference.
Knowledge graphs allow organizations to integrate data from various sources, including structured, semi-structured, and unstructured data. By connecting diverse data points, they enable a contextual understanding that enhances data analysis and interpretation.?
领英推荐
This ability to combine and link disparate data sources empowers organizations to gain a comprehensive view of their information landscape. And that's mission-critical for being able to develop, and execute, an information strategy.
Knowledge graphs also provide advanced search capabilities by leveraging relationships between entities. As a result, users can navigate through vast amounts of data in a more intuitive and efficient manner, uncovering previously hidden insights, patterns, and correlations.?
Expert Graphs
One company that stole a march on the knowledge graph market two decades ago is Bulgarian-based Ontotext. Sumit Pal, Strategic Technology Director, recently extolled the virtues of knowledge graphs on a DM Radio episode focused on the popular concept of data mesh.?
"When I was at Gartner, one of the most common questions was: Do I go with a data lake, a data lakehouse, or data mesh? And that was sort of a wrong question to ask," he said, suggesting the architecture of a data solution is secondary to the semantic value within corporate data.
"And that is where knowledge graphs come into the picture," he continued. "Knowledge graphs help you to build the context and the semantic meaning around the data that each of these decentralized data teams are building." In that sense, knowledge graphs are the glue.?
Then there's the sharing part! Data and insights generate value when shared with the appropriate parties, both internal and external. "Teams need to exchange data in a more standardized way, in a more controlled, governed way where knowledge graphs can again come in, especially those built with RDF technology."
The RDF acronym refers to the Resource Description Framework, a standard adopted years ago by the W3C consortium. Originally designed for metadata management, RDF has grown to be a cornerstone of graph database technology, amplifying the ability to infer relationships.
This becomes especially important in the fast-growing realm of data exchanges, and the wildly popular world of data products. "Data products need to interact with each other, reference each other with certain data contracts. And knowledge graphs can help you to make sure that the data contracts are adhered to as these data teams are building their products."
The Big Picture
Putting the pieces together, knowledge graphs are clearly here to stay, and will play a critical role in helping companies make the most of their information strategies. In fact, they can add value across the entire information lifecycle: from search and discovery to analysis and action.
What's more, knowledge graphs will likely help companies navigate and ultimately wield the power of Generative AI, by providing a trusted foundation of meaning which can be leveraged by all manner of algorithms. This will reduce / mitigate the down side of hallucinations.?
While concepts like data mesh are challenging the status quo, the reality on the ground is that making sense of data should always be paramount. That's the primary purpose of knowledge graphs, to facilitate the process and discipline of understanding data.?
Sharon-Drew is an original thinker and author of books on brain-change models for permanent behavior change and decision making
1 年Eric Kavanagh Merv Adrian I wonder if either of you have time to speak. I'm an inventor of systemic brain change models used in sales, leadership, coaching. I"m currently starting work on an API with SDK to formulate and prompt a self-generated sequence of brain directional questions (took me 10 years to invent) that lead users to specific neural circuits to their values-based criteria for decision making. In other words, a very different form of 'data'. Would love your thoughts, and would love to ask questions. as an author and inventor, i'm not in the ai field. Thanks in advance.
Sharon-Drew is an original thinker and author of books on brain-change models for permanent behavior change and decision making
1 年Eric: I'm an original thinker and inventor of systemic brain change models that I've been training to sellers, leaders, coaches, in Fortune 500 companies for 40 years. One of my inventions (took me 10 years!) is a Facilitative Question that is brain directional rather than info gathering: it leads to the specific neural circuits where values-based answers are stored. I'm currently figuring out how to use my FQs in ai - Facilitation.ai, maybe? or the front end of causal ai? Currently, questions are data-driven instead of criteria driven, and criteria and values are where accurate answers are stored. Then you'd have a different type of graph to work with. Stay tuned. Or help me figure out what to do with my stuff. I certainly have a place in ai, somewhere. God knows, I don't :)
AI & Data Strategy | Data Visualization + Diagramming | AI Advisory | Product Design | Tech Industry Professional
1 年Kudos for this article! Adding one thought - knowledge graphs are less about being a specific product and more about an approach to data solutions, enabling a more holistic understanding of diverse information sources.
Wandering in Latent Space
1 年ditto... great article, Eric. On the right track... Suggest you interview companies exploring the finer structures in hi-dim vector embedding space as the means of mapping semantic similarity to proper ontologies like RDF. Then, we would have the bridge of LLM to KG.
Director - Strategic Initiatives @ Cisco | Strategic Advisor @ Spearhead | Digital Transformation, Cloud GTM, AI, Finance, Operations, Servant Leader | MBA, CFA
1 年Eventually everything boils down to relationships.. great article Eric