Knowledge Graphs And Machine Learning - The Future of AI Analytics?

The unprecedented explosion in the amount of information we are generating and collecting, thanks to the arrival of the internet and the always-online society, powers all the incredible advances we see today in the field of artificial intelligence (AI) and Big Data.

With this in mind, a great deal of thought and research has gone into working out the best way to store and organize information during the digital age. The relational database model was developed in the 1970s and organizes data into tables consisting of rows and columns – meaning the relationship between different data points can be determined at a glance.

This worked very well in the early days of business computing, where information volumes grew slowly. For more complicated operations, however – such as establishing a relationship between data points stored in many different tables - the necessary operations quickly become complex, slow and cumbersome.  

Machine learning – the self-teaching algorithms designed to become more accurate at generating predictions from data as they are fed increasingly large volumes of information – often need to draw data from vast and disparate datasets. It quickly became apparent that a new approach was necessary.

The Knowledge Graph

There have been many attempts to improve on the functionality of the relational database since the model was first developed. One which is quickly growing in popularity due to its flexibility and potential for dealing with complex, interrelated data, is the knowledge graph (sometimes known as a graph database.)

The meaning of the term is not precisely set-in-stone – for example, Google has a specific feature which it calls the Knowledge Graph, which powers the section of its search results page that displays factual information, drawn from recognized sources of authority.  

While this is built with several of the same ideas that feed into the broader concept of the knowledge graph, it's not the be-all and end-all of the technology.

In basic terms, a knowledge graph is a database which stores information in a graphical format – and, importantly, can be used to generate a graphical representation of the relationships between any of its data points.

This means that the apparent advantage over the older, relational style database is that the relationships between any data points can be calculated far more quickly and with less compute power overheads, regardless of whether the data points fit neatly together into a table.

Oracle – which actually released the first commercially available relational database management system in 1978 – is now leading the field in making knowledge graph systems available to the wider business community. Currently, you're more likely to find them being used by tech giants and research institutions, but this is set to change in the very near future.

Hassan Chaffi, senior director of research and advanced development at Oracle Labs, describes the difference between relational and graph databases to me in this way: "With a relational database … it just deals with tables … it allows you to find a row in a table, or take two tables and combine them … and with every join, you're traversing one hop in these graphs, but you're reasoning about it in these tabular ways.

"So now what we're saying is, what if you were to rearrange that same information as a graph? Now it's visual, and instead of having these tables representing connections, you have vertices which represent people, or accounts, and you have 'edges' which represent relationships. Now I can more quickly say, 'ok, are Bob and Charlie related? And I can see easily that they are."

Who uses knowledge graphs?

At the moment, knowledge graphs are widely used by the tech giants that have made gathering and analyzing huge volumes of messy, complex data their core business. They power Google’s search engine, as the original page rank algorithm is based on a form of knowledge graph, as well as later additions to its search technology such as the Knowledge Graph.

Facebook also relies on this form of information organization, to keep track of networks of people and the connections between them, as well as every other data point they use to build a picture of their users, such as favorite artists and movies, events attended and geographical locations. One of its most significant breakthroughs is considered to be the realization that the relationships between data points are as valuable as the data points themselves when it comes to building social networks.

Netflix uses knowledge graph technology to organize information on its vast catalog of content, drawing connections between movies and TV shows and the actors, directors or producers who put them together. This helps them to predict what customers might like to watch next, and foster the "binge-watching" model of consumption it has built its business around. 

Electronics and manufacturing giant Siemens uses knowledge graphs to build accessible models of all of the data it generates and stores, and use it for risk management, process monitoring and building “digital twins” – simulated versions of real-world systems which can be used for design, prototyping and training.

In supply chain logistics, knowledge graphs can be used to keep track of inventories of different components and parts, allowing manufacturers to understand the crossover between materials that are used in different products.

They are also being quickly adopted by the financial services industry, where they are useful for assessing whether or not transactions are fraudulent, as well as many other functions such as marketing and investment analytics.

Chaffi tells me "In particular for crime and compliance … for money laundering, one can think of money moving around as a graph, and you need to think about whether those movements are risky or not … If you were to follow the trail that starts from one place, does it come back to the same place after an indefinite number of hops?"

Machine learning

With industries increasingly adopting machine learning, it seems likely that knowledge graph technology will also evolve hand-in-hand. As well as being a useful format for feeding training data to algorithms, machine learning can quickly build and structure graph databases, drawing connections between data points that would otherwise go unnoticed.

Machine learning is great for answering questions, and knowledge graphs are a step towards enabling machines to more deeply understand data such as video, audio and text that don’t fit neatly into the rows and columns of a relational database.

This could potentially revolutionize fields where the technology undoubtedly has applications that have not yet been fully explored, including healthcare and law.

As with machine learning itself, what started as an academic exercise before being adopted by the most cutting-edge tech companies will no doubt “trickle down”, as tools and frameworks designed to make it accessible become more widely available. 

Thank you for reading my post. Here at LinkedIn and at Forbes I regularly write about management and technology trends. I have also written a new book about AI, click here for more information. To read my future posts simply join my network here or click 'Follow'. Also feel free to connect with me via Twitter, Facebook, Instagram, Slideshare or YouTube.

About Bernard Marr

Bernard Marr is an internationally best-selling author, popular keynote speaker, futurist, and a strategic business & technology advisor to governments and companies. He helps organisations improve their business performance, use data more intelligently, and understand the implications of new technologies such as artificial intelligence, big data, blockchains, and the Internet of Things.

LinkedIn has ranked Bernard as one of the world’s top 5 business influencers. He is a frequent contributor to the World Economic Forum and writes a regular column for Forbes. Every day Bernard actively engages his 1.5 million social media followers and shares content that reaches millions of readers.