So Who Invented NLG?

I’ve been involved with natural language generation (NLG) for about 20 years now, starting when I did my PhD in this field at the Information Technology Research Institute (ITRI), University of Brighton, UK.

Nowadays, although still well behind in recognition to natural language understanding (NLU) — which people usually refer to ‘NLP’ — NLG is starting to become a more prominent area commercially.

What is NLG then, and where did it come from?

A brief history of NLG

NLG could be thought of as the reverse process of understanding a text, i.e., instead of starting with a text and getting to its understanding in terms of the concepts involved, NLG starts with a process to extract concepts from data, and then transform them so that the result will be a text that expresses those concepts.

As a field, NLG started with the simplest of the tasks, which is how to transform a small number of concepts into a sentence, a task referred to as ‘sentence realisation’. That’s about putting basic linguistic elements (words) into their correct order and making sure morphological processes (such as the verb agreeing with its subject in number) are applied. Initially this was sufficient when all that was needed was the transformation of a relatively small amount of data into concepts that eventually would become perhaps one or two sentences.

When the field got more ambitious, and the amount of information to be conveyed increased, researchers started working with structures for selecting what content (i.e., which concepts) should be expressed, and organising that content to produce fluent texts that might be a few paragraphs long.

Next, techniques were developed to address questions about referring expression generation (choosing whether a full name, a description or a pronoun should be used, or more exotic pointing words like ‘this’, ‘that’, and ‘there’) and the most appropriate choice of words came into play to allow machines to generate high quality natural language – just like we humans do.

But who invented NLG?

A question that comes to mind is: who actually invented NLG?

If only Steve Jobs was still alive! We know what he would say, don’t we? I'd say: “No, no, no, Apple, it wasn’t you!!! And, forget Siri, you wouldn’t know the answer!”

Some might ask: “Winograd?”, with his SHRDLU system (1972); “Simmons and Slocum?”, generating sentences using ATNs representing linguistic knowledge (in a paper, also from 1972)? Any o(th|ld)er references???

“No”, I’d say!!! None of the above…

The inventor of NLG was an ingenious engineer at an electric engineering company. His name was Adolph Knipe. I hear you saying: “Never heard of him, I’m afraid”. So bear with me for a second — or for a few more minutes, more probably, as I suppose you’re not a machine reading this!

Knipe had just completed the building of a ‘great automatic computing engine’, and his boss, Mr Bohlen, insisted he take some time off: “I’d like you to take a holiday, Knipe. You need it.”

And it was when at home, looking at a sheet of paper with a half-page story in the typewriter that was depressing him (more about this later) that an idea — “a delicious idea, but so impracticable it doesn’t really bear thinking about at all” — came to his mind, and “from then on, Adolph Knipe began to think about nothing else”.

The Great Automatic Grammatizator engine

Knipe’s idea was of an “engine” that could “arrange words in their right order according to the rules of grammar. Give it the verbs, the nouns, the adjectives, the pronouns, store them in the memory section as a vocabulary, and arrange for them to be extracted as required. Then feed it with plots and leave it to write the sentences.”

Knipe was a frustrated amateur writer whose stories were repeatedly rejected by the magazines. With his new engine, he envisioned that he’d be able to produce narratives that would suit the different styles required by each particular publisher. He even explained to Mr Bohlen some of the refinements that would make the reader think that the writer was “very wise and clever”. The trick was to insert “at least one long obscure word into each story” and the engine would have “a whole stack of long words stored away just for this purpose”.

Mr Bohlen would then ask: “Where?”

“In the ‘word-memory’ section, he said, epexegetically.”

Eventually the engine evolved to such a state that it was possible to write books with it. There were so many forms of control that “it looked like the instrument panel of some enormous aeroplane” and the process looked “rather like flying a plane and driving a car and playing an organ all at the same time”.

Conclusion

So there you have it: Adolph Knipe.

What you might have not yet realised is that he was actually the main character of a story, ‘The Great Automatic Grammatizator’, written by Roald Dahl and published in the book ‘Someone Like You’ in 1953.

And now you know who really invented NLG!

Acknowledgment

Thanks to Nadjet Bouayad-Agha who introduced me to Dahl’s text almost 20 year ago!