Generative AI is a type of artificial intelligence that can create new content and ideas, including conversations, stories, images, videos, and music.

Understanding FM functionality

The size and general-purpose nature of foundation models make them different from traditional ML models. FMs use deep neural networks to emulate human brain functionality and handle complex tasks. You can adapt them for a broad range of general tasks, such as text generation, text summarization, information extraction, image generation, chatbot, and question answering. FMs can also serve as the starting point for developing more specialized models. Examples of FMs include Amazon Titan, Meta Llama 2, Anthropic Claude, AI21 Labs Jurassic-2 Ultra, and more.

Self-supervised learning

(opens in a new tab)Although traditional ML models rely on supervised, semi-supervised, or unsupervised learning patterns, FMs are typically pretrained through self-supervised learning. With self-supervised learning, labeled examples are not required. Self-supervised learning makes use of the structure within the data to autogenerate labels.

Natural language processing (NLP)

(opens in a new tab)NLP is a machine learning technology that gives machines the ability to interpret and manipulate human language. NLP does this by analyzing the data, intent, or sentiment in the message and responding to human communication. Typically, NLP implementation begins by gathering and preparing unstructured text or speech data from different sources and processing the data. It uses techniques such as tokenization, stemming, lemmatization, stop word removal, part-of-speech tagging, named entity recognition, speech recognition, sentiment analysis, and so on. However, modern LLMs don't require using these intermediate steps.

Recurrent neural network (RNN)

RNNs use a memory mechanism to store and apply data from previous inputs. This mechanism makes RNNs effective for sequential data and tasks, such as natural language processing, speech recognition, or machine translation. However, RNNs also have limitations. They are slow and complex to train, and they can’t be used for training parallelization. To learn more about the performance capabilities and functionality of RNNs, refer to the "Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network" article at the end of this lesson.

Transformer

A transformer is a deep-learning architecture that has an encoder component that converts the input text into embeddings. It also has a decoder component that consumes the embeddings to emit some output text. Unlike RNNs, transformers are extremely parallelizable, which means instead of processing text words one at a time during the learning cycle, transformers process input all at the same time. It takes transformers significantly less time to train, but they require more computing power to speed training. The transformer architecture was the key to the development of LLMs. These days, most LLMs only contain a decoder component. To learn more about transformer architecture, refer to the "Attention Is All You Need" article at the end of this lesson.

Text-to-image models

Text-to-image models take natural language input and produce a high-quality image that matches the input text description. Some examples of text-to-image models are DALL-E 2 from OpenAI, Imagen from the Google Research Brain Team, Stable Diffusion from Stability AI, and Midjourney.

To learn more about text-to-image models, specifically diffusion architecture, review the following slide.

Large language models are a subset of foundation models. LLMs are trained on trillions of words across many natural language tasks. LLMs can understand, learn, and generate text that’s nearly indistinguishable from text produced by humans. LLMs can also engage in interactive conversations, answer questions, summarize dialogues and documents, and provide recommendations.

Because of their sheer size and AI acceleration, LLMs can process vast amounts of textual data. LLMs have a wide range of capabilities, such as creative writing for marketing, summarizing legal documents, preparing market research for financial teams, simulating clinical trials for healthcare, and writing code for software development.

Understanding LLM functionality

As you learned earlier, most LLMs are based on a transformer model. They receive the input, encode the data, and then decode the data to produce an output prediction.

Neural network layers

Transformer models are effective for natural language processing because they use neural networks to understand the nuances of human language. Neural networks are computing systems modeled after the human brain. There are multiple layers of neural networks in a single LLM that work together to process input and generate output.

LLM use cases

You can use LLMs for a wide range of tasks and in almost every domain. To learn more about LLM use cases, choose each of the numbered markers.

New use cases will arise as LLMs evolve and gain a broader audience. Generative AI will play a transformational role in every industry.?

Resources

Getting Started with Generative AI and Foundation Models

To learn more about foundation models from an AWS whitepaper, choose the following button.

GEN AI WHITEPAPER

What Is Natural Language Processing (NLP)?

To learn more about NLP on the AWS website, choose the following button.

WHAT IS NLP?

Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network To learn more about RNNs from a scholarly article by Alex Shertinsky, choose the following button.

RNNS ARTICLE

Attention Is All You Need To learn more about transformers from a scholarly article by Ashish Vaswani and others, choose the following button.

TRANSFORMERS

High-Resolution Image Synthesis with Latent Diffusion Models To learn more about diffusion from a scholarly article by Robin Rombach and others, choose the following button.

that's wrap up for today!