How do computers read at superhuman scale?

Introduction 

I wanted to write a short piece to answer some common client questions I get asked:  

• “How is it a computer can read millions of articles, company descriptions and patents in the time it takes me to order a coffee?”
• “How does it actually work? “
• “Can I trust the results?”
• “How can it help me in my business?”.

This article is intended as a non-technical guide to explain how computers read and how you can apply this to your business and your team today. It is based on first hand perspectives, rather than in depth research, but I hope it inspires you to look at how the new reading capabilities that Natural Language Processing (NLP) provides can create value in your organization.  

Q: Remind me again what do you mean by Artificial Intelligence (AI)?

If we all had a dollar for every mention of AI then we could feed the world and have change left over. Cheaper computing power, developments in approaches to writing software such as machine learning and the abundance of data are all factors that have given rise to the seeming ubiquity of the term AI and potential of AI. 

The term was coined by John Macurthy and his 1956 Dartmouth Summer Research Project on Artificial Intelligence. His objective then was “to write a calculator program which can solve intellectual problems as well or better than a human being”. The team that came together that summer worked in the belief that in the future “every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.”.  

Following their groundbreaking conceptual project it has become common to divide intelligence in this context into categories relating to practical applications such as: Reasoning, Knowledge Representation, Planning, Natural Language Processing, Perception and Generalized Intelligence. This article focuses on one of those, Natural Language Processing (NLP) and explores examples of what is now termed narrow AI ie AI focused on a specific task. 

Q: In layman's terms, how does NLP work?

Before explaining how computers read it is worth trying to think about we humans do it! Essentially we are processing words in sentences by first looking at their syntax. Syntax is the process by which words are arranged in a sentence to give it meaning and be grammatically correct.  

Current wisdom suggests in simple terms that when we read and understand sentences we are applying two main techniques. Firstly, what are called syntactic techniques, these are used to break down paragraphs and sentences to understand their syntax, for example word or text segmentation involves breaking a paragraph down into its constituent parts. Secondly semantics, which is about trying to understand meaning from the language, this is a very complex task and includes among other things understanding entities (what and who is being talked about), context and intent of the words. So for computers to read they need to start to tackle both syntactics and semantics.  

Q: How can a computer read?

As you know at their heart computers fundamentally convert everything to a 1 or a 0 (zero). When you type, the keyboard is converting the keystroke to a code, you may be familiar with the acronym ASCII which stands for American Standard Code for Information Interchange and the later Unicode Standard, these are the basic code books which converts letters and symbols to unique numbers (eventually 1 and 0’s).  

Of course it works even if you have not typed it but is ingested as a document into a computer, in this way a computer can take any letter, word or symbol and by using the code book “read” it. This is how a very basic computer search, or find function, works. It is looking for words, perhaps combinations of words, matching them and returning results. Think of this like “hits”. 

However NLP is much more than “hits” and involves deeper reading of sentences and blocks of words, for example in documents, for meaning. Data sitting in books, news articles or blogs is known today as unstructured text data. That is because although it has some structure it is not in a database tagged and filed neatly into sections but exists in natural sentences.  

It has been estimated by IDC that unstructured data will account for 80% of ALL the world’s data within 5 years. 

This is the domain of NLP.

Q: How does a computer take unstructured text data and make it structured

For computers to go beyond basic word “hits” and carry out NLP they first need to be able to take chunks of text, we call them a corpus of text, and turn it into a structure for analysis. To do this words and sentences are turned into an ordered set of values which a computer can analyse.  

One technique for this is called creating vectors. Think of a vector as a set of values that has been put in a sequence or order. Here is an example courtesy of my Quid colleague Will Mees. Imagine you were given 5 documents to read and they have the following text in them.

Now the computer can convert these to a vector (ie a list in order), by counting the words and placing them into a table as follows:

In this example the computer only considered single words like “dog”, this is what is called a unigram. This analysis could be extended (ie more columns) to include phrases with two words like “drink water”, which are known as bigrams (“n”-grams where “n”=2 or bi) and so on. In this way unstructured text data is becoming turned into structured data.  

Once you have vectors and n-grams you can start to look for patterns, for example you could ask which documents are most similar to each other. To do this you could plot the results on a graph with two axis and see which ones are closest together and which are furthest away, this would identify outlying documents or paragraphs and highlight very similar ones.  

With a large corpus you would need some additional techniques to group or cluster similar documents with each other, for example the Louvain statistical technique, which allows you to delineate communities based on how similar those documents are to documents in another community.

With text data in a structured format it then becomes possible to begin some aspects of what the human brain is doing with NLP. You can parse, or break up, sentences to look for meaningful phrases within them. It is also possible with other techniques to extract or connect entities or meaningful items within the text such as: people, places, brands, companies, numerical expressions and so on.

Q: What makes it challenging?

There are three things in my view that make NLP particularly hard context, language quirks and volume.

Firstly context, you may have seen this example before but it beautifully illustrates how our brains do something so easily that computers find challenging and that is context: “Paris Hilton stayed at the Paris Hilton.” What are they talking about? Or knowing that Donald Trump and Trump are actually the same person. Or what if the meaning of the sentence only becomes apparent at the end? “The jeans label said they were long lasting, really comfortable and great value for money. However mine suck!” Or the meaning is not as positive as you first think: “I gained 10 pounds after my wedding day!”. 

Secondly everyday natural language quirks like: idiom; sarcasm; subtlety or indeed literalism. “Coke adds life” does this mean “Coke raises your ancestors from the dead”?. 

In NLP the algorithms concerned with solving these problems include named entity extraction (which people, places etc), coreference resolution (which words are referring to the same object), part of speech tagging , sentiment analysis (happy or angry?) and sense disambiguation.  

Lastly volume, for business applications scale and response time is everything, so the ability to process huge volumes of text data rapidly and efficiently is currently testing the ingenuity of the leading NLP data scientists, software engineers and cloud providers to their limits. They have been dramatically helped by a shift in programming approaches from structured programming to using data and deep learning techniques, such as neural networks, combined with cheaper and faster processing power. Of course the text item itself needs to have an appropriate volume itself to provide meaning as one or two words alone in a social media exchange are unlikely to provide insights of value.

Q: What NLP capabilities are being applied in enterprises today? 

If we look at what is being written about NLP in the news and in blogs to give us an indication of what is resonating by using Quid’s NLP platform we can see there were 2886 stories where NLP was the core focus over six months January - June 2019.

The largest clusters by volume of articles were related to: AI Chat Bots (12%), Health and Social Care (9.2%), General applications of NLP (8.9%), Deep Neural Networks (8.6%), Data Science Symposium (7.5%) & Customer Experience Analytics (7.3%). Source: Quid.  

If we focus more on the shakers and movers in the AI and NLP world for some ideas and ask Quid who the Key Opinion Leaders are in NLP over the last year (it read 4912 articles published in the last 12 months, extracted 531 most relevant ones and ranked the people mentioned in them) it tells us:

Yann Lecun, a New York University professor, who also works on AI at Facebook has the highest relevance score and an overall rank of 1. He was one of the winners in March 2019, (alongside Yoshua Bengio (#2) and Geoffrey Hinton ((#5) who won this year’s Turing Award, presented by The Association for Computing Machinery (ACM), sometimes referred to as the Nobel Prize of computing. As ACM President Cherri M. Pancake commented on their contribution to AI and NLP while giving the award “The growth of and interest in AI is due, in no small part, to the recent advances in deep learning for which Bengio, Hinton and LeCun laid the foundation...Anyone who has a smartphone in their pocket can tangibly experience advances in natural language processing and computer vision that were not possible just 10 years ago.”

From a personal professional perspective I see NLP capabilities are currently being deployed by commercial enterprises in three main areas.  

The first is language pattern and embedded signal recognition. This area is concerned with reading text to identify broad patterns or signals, Given the desire to understand and even predict consumer behaviour it is often used for consumer and market trend spotting. In this case NLP algorithms will help you understand the volume and time series of mentions and social resonance of topics and companies in the news, social media or reviews. In this way companies like Walmart are able to gain insights into the relevant trends affecting their 140m weekly customers [Understanding consumer trends and the American Family Link].

The same principles can be applied to understanding almost any set of consumers, for example it is now being used extensively to understand the voice of the patient in healthcare as UCB have shown in their extensive study in NEJM. [UCB - Understanding the Voice of the Patient in chronic illnesses: Link].  

Given language is convening signals about many sorts of activity it means that publicly available data can also provide insights about events. For example imagine the signals in articles highlighting a CEO resigns, or a factory closes or a supply chain is disrupted. This is being harnessed by those interested in event and anomaly detection.  

Although these signals are not predictive in themselves companies like ING are building models to translate these into early warning signals for financial performance or risk factors, this is a growing use case for NLP [Early Warning Signals for Credit Risk Link.] The same principles can be applied to the M&A due diligence process, as demonstrated by the PwC Deals Team [PwC - Supporting deals with augmented intelligence Link].  

It will also help identifying outliers in written documentation such as case notes, call centre logs or automate the review of property risk engineering surveys as AXA Insurance have done. [AXA XL implemented natural language processing (NLP) Link]. 

By reading at scale what executives and observers are actually saying innovative companies are enhancing their competitive intelligence and strategy prediction capabilities. It also means topic research for fast moving and complex areas is transformed, BCG’s recent blockchain analysis report is a case in point [BCG - Researching complex and fast moving topics, Blockchain Link].

The second area is language interpretation. This is similar to pattern recognition but with a focus on creating defined actions. The most common example converting spoken voice to text then executing a task. For example asking Google/Siri/Alexa/Cortana to create a calendar entry, asking your remote to find a TV show [Comcast Link], or using voice commands in a vehicle to control features and functions havde come a long way since the Honda Acura in 2005 [Link].

Lastly is language generation, which is concerned with automatically creating language in context. We have become used to predictive text, grammar and style suggestions in our day to day use of devices and word processing software. More recently chat bot conversations are delivering this capability and more intelligent responses to customer service and other interactions. Also areas such as legal contract drafting are subject to advances in NLP as well as summarising a large volume of text to highlight the most interesting or relevant ideas contained in them.

Q: What are the primary problems companies are using NLP for?

If we assume NLP is directionally similar in its applications with other aspects of AI the Deloitte 2018 study of US executives tells us that key benefits lie in enhancing existing products, optimising internal operations and making better decisions. 

Here is the full list of the % who rated these areas as a top-3 benefit from applying AI:

The scale of the implementation ambition can be seen by PwC’s 2019 research report which said a full 20% of organizations surveyed plan to implement AI enterprise-wide in 2019. 

As you can imagine these NLP capabilities can be applied in a myriad of processes and use cases within the enterprise and across functions and teams. At a very macro level primary uses are of course related to where language and text contains high value meaning or signals and the scale or complexity of reading the texts makes it hard for humans to keep up or find the signals at all. 

One key problem NLP also addresses is that it helps reduce cognitive bias, Wikipedia lists ~200 common types of bias [Link] which we all “suffer” from, NLP addresses many, for example our tendency to anchor on one piece of information when making decisions, usually the first piece of information acquired on that subject.

These macro areas often relate to understanding customer or patient needs, predicting global trends or researching fast moving technical areas where the volume and richness of the text makes it impossible to read it all. So this leads to applications in PR, marketing and new product development, such as understanding weak signals hidden in voice of the customer text (such as reviews or blogs) and then creating text content that will resonate most effectively or new products and services that will address unmet needs. You can also now read all the marketing material produced by your competitors and see if they are doing a better job than you are! It also allows you to run a very unstructured survey process with open ended questions such as “What do you think of this product?”, which in the past you would never have considered. If you are running a marketing campaign, lobbying or organising a conference having reliable statistics about who the actual key opinion leaders are in a space based on what they have written and how it is resonating becomes vital. 

Research and strategy teams need to keep up with, or even predict, trends and competitors moves. This involves assimilating and analysing signals or outliers buried in research papers, published reports or job adverts, NLP allows that to happen at scale and in real time. For healthcare companies interested in understanding the voice of the patient NLP has become a game changer in understanding what people really want from their healthcare experience.  

In sales and operations teams NLP is being applied to improve customer service and efficiency, For example in areas such as call log or call centre transcript analysis, driving revenue by trying to improve conversion ratios, reducing costs by looking for predictable maintenance patterns or root cause analysis and even helping retailers reduce returns. It is also the enabling capability to reduce admin effort or robotic process automation in processes with high volume of document or text processing, for example lending forms or basic applications or referring fake news to human fact checkers. 

In finance related processes there are extensive applications from assessing creditworthiness, due diligence on companies you are about to buy, through to compliance monitoring and risk sensing. HR teams have found uses for NLP in understanding voice of the employee through feedback surveys or analysing the open source intelligence held in the sentiment in public posts on sites such as GlassDoor or Indeed.com, it also helps screen CVs and matches applicants to vacancies. IT departments can use it for structuring requirements gathered in systems development processes and Digital Transformation such as Design Thinking workshops or agile development. 

These capabilities are of course also being employed at mass scale for end products and services with language at the core of the user experience. From early uses in spam filters, auto-correct and auto-complete it is now almost ubiquitous in device control (e.g. phones, connected home devices, smart speakers), intelligent chat bots and even medical apps, reading healthcare records. It is also beginning to be used in highlighting unwanted text, for example in identifying so called fake news and in social media highlighting potentially harmful or damaging posts. Wherever you see unstructured language being used in human to machine interactions NLP will not be far behind.

Q: Where will NLP go next?

I decided a while ago never to try and predict the future, I particularly like Nile Bohr’s (Nobel laureate in Physics) take on this: "Prediction is very difficult, especially if it's about the future."

The recent step change improvement in machine learning approaches, where the use of training data sets and deep neural networks for example have proved a significant boost to NLP capabilities as well as other AI applications so one thing we can be certain of is that NLP is advancing very quickly.

From a personal perspective I see trends which suggest momentum in NLP in the following areas. In language pattern and embedded signal recognition, the ability to automatically identify and alert you to relevant events and signals in unstructured text is emerging. With language interpretation moving beyond just instructions foreign language simultaneous translation seems tantalizingly close especially with Google’s recent investments and success with Google Translate. I sometimes ponder on how this may one day bridge into the human to animal communication sphere, Dr Dolitte may be closer than you think! For language generation the work on legal contract drafting is making great strides, although I have been less impressed with NLP developers efforts in genuinely creative content creation, Japanese Haiku’s poetry aside. Link

To sum up, NLP is here to stay and will become an increasingly useful asset in the corporate world driven by the axiom that the more information you have the better decisions you are likely to make. NLP will help you make sense of signals and insights that you were not able to find as a human being simply by reading, it will also encourage you to ask the difficult questions that structured data often does not give a complete answer to, such as “What do my customers really want?”, and at the end of the day will help save your precious time for reading what really matters.

Appendix - References and further reading

ACM award

AI powered drawings courtesy of Autodraw

Dartmouth 1956 TOR

Deloitte: AI and intelligent automation business survey

Forbes: Where does AI fit in the modern workplace?

Forbes: 4 NLP Techniques to increase your understanding

Deep Learning for NLP - Overview of recent trends

Deep Learning Tutorial - Geoff Hinton, Yoshua Bengio & Yann LeCunnfoWorld: NLP explained

IDC: Unstructured Data at Scale

PwC: 2019 AI Predictions 

Quid Inc

Solutions Review: 80% of your data will be unstructured

Background on Tony Maile 

I specialize in helping senior leaders unlock value from applying AI and am leading Quid’s consulting vertical globally, working with the likes of BCG, PwC, Deloitte and Accenture. I am seeing Quid’s advanced NLP platform being used to win new business, create new service offerings and drive best practices in many of the leading strategy and management consulting firms. Prior to this I led our Enterprise business with clients such as Walmart, NASA and INTEL. Before Quid I was a partner in IBM's consulting business and a leader in IBM’s cognitive Watson business.

If you would like to continue the conversation please get in touch:

www.dhirubhai.net/in/tonymaile