Demystifying Generative AI : Generative AI Models(Part-3)

After providing a brief overview of AI/ML fundamentals in Part 1 and delving into the fundamentals of Generative AI in Part 2, it's now time to dive deeper into the field. In this last part of trilogy of generative articles,I will explore vital architectures and models used in Generative AI. Additionally, I will discuss the importance of prompt engineering and highlight different services available for developing generative AI applications.

Deep Learning Models - Here, Just to make things simple, I will talk about these models from the conceptual point of view, as for detail internal understanding of each model required more specific discussions.

Particular in deep learning, it has following models however these are not specifically Generative AI models, but Generative AI use those as supportive models specifically for more specialised tasks on labeled or reinforcement dataset.

Artificial Neural Network (ANN/MLP)- These Model neurons work together to process information and make predictions. It's like a group effort where each neuron contributes to understanding patterns in the data and making decisions.

Convolutional Neural Network (CNN)- as a special type of neural network that excels in understanding images. It mimics how our brain processes visual information. It breaks down an image into smaller parts, analyzes them, and then combines the results to recognize objects or features in the image

Recurrent Neural Network (RNN)- is a type of neural network that is good at understanding sequences of data, like time series or sentences. It keeps track of the information it has seen previously and uses it to understand the context and make predictions. It's like having a memory that helps understand the current situation based on what came before.

Generative AI Models-

Generative AI has primally three (there are other models as well) famous models for working . They are General Adversarial Networks (GANs), Transformers and Auto-encoders-.

Transformers -

As we talked of LLM in PART 2 , I am picking transformer first, as LLM and transformer are strongly connected., Think of the Transformer model as a master storyteller. It can take a sentence or a sequence of words as input and generate a meaningful and coherent output. Unlike other models that process data sequentially, the Transformer model considers the context of each word in relation to all the other words in the input sequence.To do this, the Transformer model uses Attention mechanisms . Following image is elegant example of fundamental LLM processing with transforms. One can see , for LLM , Tokenising and Token embeddings are crucial .

Let's discuss little about attention mechanisms.

?The Attention mechanism is a technique employed in language models to comprehend the relationships between words in a sequence. This mechanism enables the model to focus on different parts of the input and assign weights or scores to each word based on its relevance to the current context.

By incorporating the attention mechanism, language models can effectively capture connections between words, even when they are distant in the sequence. Consequently, this improves the model's capability to generate outputs that are contextually relevant.

The Transformer model has been a game-changer in generative AI because it can generate high-quality text, make accurate translations, or even create realistic dialogue. Its ability to consider the broader context and dependencies within a sequence has made it a go-to choice for various text related generative tasks.

Auto-encoders-

The Autoencoder has two main parts - an encoder and a decoder. The encoder takes the input data, like images or text, and compresses Then, the decoder takes this compressed representation and tries to recreate the original data.

Imagine you have a picture of a dog. The encoder learns to capture the important features of the dog, like its shape and color, and encodes them into a compact representation. The decoder then uses this representation to generate a new picture of a dog that looks very similar to the original.

Auto-encoders are useful for tasks like image generation, demonising, and dimensionality reduction, image identification and reconstruction.

General Adversarial Networks (GANs)-

In GANs, there are two parts: the generator and the discriminator. The generator task is to create realistic-looking data, such as images or sounds. The discriminator job is to determine whether the created data is real or fake.

The generator and discriminator play this game together. The generator tries to make its creations as realistic as possible to fool the discriminator, while the discriminator tries to become better at distinguishing real from fake. This back-and-forth competition pushes both the generator and discriminator to improve over time. This competition is called a Generative Adversarial Network (GAN).

As a result, GANs can generate high-quality, realistic data that can be used for various purposes like creating lifelike images, synthesising music, or even generating human-like text. GANs have opened up exciting possibilities in the field of artificial intelligence and creativity.

There are so many other models in generative AI, however, I just conceptually talked about some of the most important and popular models.

Prompt Engineering -

It is important to highlight the significant role of prompt engineering. In this context, the prompt serves as the means to interact with the model and pose questions. It refers to the text that is provided to the model along with the query and supporting evidence, which can include new or related facts.

Similar to how humans require context to provide better answers, models also need context and additional information about the query. Therefore, having a well-crafted prompt becomes extremely crucial to obtain accurate and appropriate responses from the model. Following are some of the best practices for prompt engineering.

Development with LLMs-

Numerous tech giants have developed their own large language models (LLMs), resulting in a diverse range of models in the field. These models vary in their availability, with some being open source while others remain proprietary. Notable examples of proprietary models include BERT and GPT, while open-source alternatives like Dolly from Databricks and BART from meta have also gained prominence. Microsoft recently released ORCA in open source market.

One can access these models directly, with the help of respective platform services.

Generative AI services -

Every CSP/ Platform has their own model (LLM or others) and have specific services to interact with model and framework /studio. Like Google have PaLM API for interacting with its LLM. Please have a look on below image. There are the services from diffrent companies by using these, users can create Images, Text and Generate Code etc, There are few framework or platform related services as well. Like Generative AI studio, it is providing interface for Generative AI development. CoPilot is Code generation service can act as a co-developer.

LLMs, being pre-trained models, eliminate the need for the complex and time-consuming aspects typically involved in regular or classic ML development.

Interacting with the respective LLMs has been made easier with the availability of APIs. For instance, to interact with the Palm LLM, developers can simply utilize the PaLM API. The code snippet below demonstrates two examples:

In the first example, the developer generates text by calling the PaLM API's generate_text endpoint.

In the second example, the Palm LLM's chat function is used to develop a chatbot application, leveraging the power of the language model."

In conclusion, I have discussed some fundamental aspects of Generative AI in these trilogy of articles (Part1, Part2, and this one). However, it is important to recognise that there are numerous other aspects to consider. From the perspective of Generative AI, it is undeniably transforming our world. In my upcoming write-up, I will delve deeper into the ways in which Generative AI is reshaping various domains and revolutionising the way we perceive and interact with technology.