Natural Language Generation

Generating new text from a given input (text,paragraphs,images,videos etc) is a new requirement in the tech world. All applications of AI or non-AI needs language generation technique to mitigate the communication challenges on all the areas.

NLG is a sub-component of

Language Creation

Abstractive Text Summarization

Machine Translation

Dialog generation in chatbot.

Creative Writing

Free-form Question Answering (Open Domain or Closed Domain based)

Image Captioning

Let's talk about it in mathematical terms. The language modeling is basically prediction of the next word given a set of words. The probability distribution is produced by the system known as Language Model.

Probablity of every word in the vocabulary. A language model is at the core of many NLP tasks, and is simply a probability distribution over a sequence of words:

probability distribution over a sequence of words:

Conditional probability of the next word in a sequence:

Let's assume we have below sequence of words:(He , is ,selfish,like, me) and we would guess the final word.A language model would estimate the probability for every word in the vocabulary:

Some potential NLG Tasks

Text summarization ( Abstractive)

Dialogue

Conversational Q&A

Story Telling

Poem Writing

NMT

Further reading

By necessity, a lot of details have been glossed over, and some of the finer points may be confusing. Here are some helpful links to fill in the gaps.

Deep learning for NLP:

• Stanford course CS224n: Natural Language Processing with Deep Learning

Reccurent neural networks:

• Understanding LSTM Networks
• The Unreasonable Effectiveness of Recurrent Neural Networks

seq2seq models:

• A ten-minute introduction to sequence-to-sequence learning in Keras
• Translation with a Sequence to Sequence Network and Attention

Neural text generation

• Neural Text Generation : A Practical Guide