What is gen AI? Plain and simple.

Recently, I was invited to mentor a group of employees at Intel Corporation in the Generative AI track. As I prepare various topics for our group discussions, I’m receiving some fundamental and exciting questions from my mentees. I’m learning a lot from these interactions, and I feel it’s valuable to share these insights with my network of people who are enthusiastic about learning the basics of AI and applying it to everyday tasks.

One of the interesting questions that I received was:

“In the context of Language Models, how do you differentiate between generative AI and classical AI?”

Language Models (LMs) are a cornerstone of modern Natural Language Processing (NLP), enabling various applications such as chatbots, translation services, and content generation. Broadly, Large Language Models can be classified into two types: Causal and Non-causal models. When experts talk about Generative AI, they often implicitly refer to causal language models. Understanding the differences between these two types of models is essential for grasping their applications and implications in various domains.

Causal Language Models (Gen AI)

Causal language models, also known as autoregressive models, generate text by predicting the next word in a sequence based on the preceding words. These models operate in a unidirectional manner, processing input from left to right. A prominent example of a causal LLM is Open AI’s GPT series.

How Causal Models Work

In a causal model, the probability of each word in a sentence is computed based on the words that have come before it.

This approach allows the model to generate coherent and contextually relevant text, as each word is influenced by the context provided by its predecessors.

There are multiple use cases of causal models or Gen AI:

1. Text Generation: Causal models excel in generating coherent and contextually appropriate text, making them ideal for applications like storytelling, article writing, and conversational agents.
2. Autocompletion: These models can be used in writing assistants and code editors to predict and suggest the next word or line, improving efficiency and reducing errors.
3. Chatbots: Causal models are well-suited for real-time conversation, as they can generate responses based on the immediate context of the dialogue.

Non-Causal Language Models (Non Gen AI)

Non-causal language models, also known as bidirectional models, consider the entire context of a sentence simultaneously rather than processing it in a single direction. The Bidirectional Encoder Representations from Transformers (BERT) by Google is a well-known example of a non-causal LLM.

How Non-Causal Models Work

Non-causal models leverage both past and future context to understand and generate text. This bidirectional approach allows the model to capture more nuanced and comprehensive representations of language.

Here, the “context” refers to both preceding and succeeding words, enabling the model to generate a richer understanding of the sentence structure and meaning.

Here are some use cases of non-causal models:

1. Text Classification: Non-causal models are highly effective in tasks like sentiment analysis, spam detection, and topic classification, where understanding the full context of a sentence is crucial.
2. Named Entity Recognition (NER): These models can accurately identify and classify entities within text (e.g., names of people, organizations, locations) by considering the surrounding context.
3. Machine Translation: The bidirectional nature of non-causal models makes them suitable for translating text from one language to another, as they can account for context from the entire sentence.
4. Question Answering: Non-causal models excel in answering questions based on a given passage of text, as they can utilize the full context to extract relevant information.

Both causal and non-causal language models have distinct architectures and use cases, catering to different needs in the field of NLP. Causal models, with their unidirectional context processing, are well-suited for generative tasks (Gen AI) and real-time applications. In contrast, non-causal models, with their bidirectional understanding are best in comprehension tasks that require full context awareness. Understanding these differences is key to leveraging the right model for the right application, thereby enhancing the effectiveness and efficiency of Natural Language Processing solutions in various domains.