What are Generative AI and LLM?

Have you heard about ChatGPT, Midjourney, Dall-e, and other tools to generate text, images, and videos? Have you heard about AI tools replacing humans at work?

Many questions have been raised with the explosion of AI technologies these days, that’s why it’s important to know what is all about it, how we reached this point, and what is behind all those amazing tools we are seeing these days.

First, let’s talk about Artificial Intelligence. What is AI? The official concept, born back in 1955 by computer scientist John MacCarthy and other fellow co-workers, describes the science and engineering of making intelligent machines. So today, I believe it’s fair to say Artificial intelligence (AI) refers to the simulation of human intelligence in machines that are programmed to “think” like humans and mimic their actions and/or processes. Machines can learn to autonomously make decisions and carry out actions on behalf of human beings.

With the AI concept, the paradigm for coding changed. Machine Learning (ML) is raised as a sub-field of Artificial Intelligence to provide a series of input data and the expected responses to obtain an output, a series of rules that allow us to obtain the correspondence between the input and output data. With ML, Deep learning was created to mimic the human brain through what is called Neural Networks architectures or artificial Neural Networks.

Machine learning models use statistical techniques to help the machine “learn” how to get progressively better at a task without necessarily having been specifically programmed for it. Instead, Machine Learning algorithms use historical data as input to predict new output values through 3 sub-categories that will help us resolve different problems:

• Supervised Learning: uses labeled datasets and correct outputs to train learning algorithms how to classify data or predict an outcome, like a Teacher-Student relationship. For example, if a system is required to classify fruit, it would be given training data such as color, shape, dimension, and size. Based on this data, it would be able to classify fruit.
• Unsupervised Learning:?it doesn’t use labeled datasets. The models work on their own to uncover the inherent structure of that data, but human intervention is still required to validate the output variables, depending on the intended use of the data and if it makes sense for the data to be utilized at the end. It’s typically used to uncover insights from massive volumes of data, detect anomalies or make recommendations.
• Reinforcement learning: it’s about taking suitable action to maximize reward in a particular situation, and it also differs from supervised learning. The input data set has no answers, but the reinforcement agent decides what to do to perform the given task; it’s bound to learn from experience. For example, if you like to train a dog at home, the agent would be the dog, and we can get the dog to perform various actions by offering incentives such as dog biscuits as a reward so the dog will follow the instruction to maximize its rewards and hence will follow every command and might even learn a new action, like begging, all by itself.

On the other hand, Deep learning is a type of machine learning that runs inputs through a biologically inspired neural network architecture which contains several hidden layers through which the data is processed, allowing the machine to go “deep” in its learning, making connections and weighting input for the best results.

There are different architectures with their own strengths and weaknesses, but the most common you’ll see out there are:

• Convolutional Neural Networks (CNNs):?often used for image recognition and classification tasks by applying filters to extract features that will be passed through a series of layers to produce a classification output.
• Recurrent Neural Networks (RNNs):?these are commonly used for sequential data and work by processing one element of the input sequence at a time and using the previous state of the network to inform the processing of the current element. RNNs are useful for language modeling, speech recognition, and sentiment analysis tasks.
• Generative Adversarial Networks (GANs):?used for generative tasks, such as image synthesis and text generation, by training 2 neural networks, one to generate fake data (Generator) and the other to discriminate between real and fake data (Discriminator). The 2 networks are trained in opposition to each other with the goal of improving the generator’s ability to generate realistic data.
• Transformers:?designed specifically for natural language processing tasks by using self-attention to selectively focus on different parts of the input sequence, allowing them to process long sequences of text efficiently so it can be used for language modeling, machine translation, and question answering.

All the architectures, models, and concepts explained before will help us better understand what Generative AIs and Large Language Models (LLMs) are, topics that are popping up often.

Generative AI is a type of AI that involves the use of deep learning algorithms to generate new data similar to a given dataset. Generative Neural Networks (GANs), autoregressive Models, and Variational Autoencoders (VAEs) basically do?Image generation?like realistic objects, animals, people, etc, for areas such as gaming, art, and virtual reality. We also find?text generators?such as coherent and meaningful texts like stories, poetry, and new articles, and going further. We can probably say that a combination of several models can give us the ability to create tools like chatGPT where humans can feel a more natural interaction, like if you are talking to another person.

At this point, LLMs come into play, especially to understand and generate natural language. The secret is these models are trained on massive amounts of text data from different sources to predict the probability of the next word in a sequence based on the previous words. This process is repeated millions of times, allowing the model to learn the patterns and relationships between words and phrases in the language it’s trained on. Ring a bell? Probably the answer would be positive since Transformers works like that.

Once the model is trained, it can be used for a variety of tasks, such as GPT-3 (Generative Pre-Trained Transformer-3) has been shown to generate high-quality human-like text across a range of domains.

The success of LLMs has been driven by a large amount of data and powerful computing resources, enabling researchers to train models with billions of parameters, allowing models to understand complex language structures to generate coherent and fluent text that is often difficult to distinguish from text written by humans, something that in the past were preventing AI to evolve faster.

All that sounds exciting, right? however, we need to be aware of some ethical implications and responsibilities of the use of all those technologies, such as the potential of those models to generate biased or harmful content and the possibility of their misuse in areas like disinformation campaigns, cyber-attacks, and what is called deep fakes, where basically anyone can impersonate and talk like other people or even create images of real people doing inappropriate things, which can lead to different real problems.

Also, the privacy of the data is something we need to have at the top of our list. We cannot share sensitive data until we are entirely sure this will be secured, and finally, have you asked yourself who should be the owner of the images, text, or any other stuff generated by the AI? Well, it’s a topic that is being discussed in many forums to set the right responsibilities and law updates accordingly.

Lastly, you also need to be aware that AI is not replacing you at work. They don’t have intentionality, they don’t have human understanding, and they don’t have instincts, yet; today, it’s mostly used to optimize our work, to perform several tasks more efficiently, but it’s also a great opportunity for all of us to evolve and keep learning how is the best way to use those technologies in our favor.

I've heard this phrase lately, not sure who is the author but I truly believe in it:

"We are not going to be replaced by AI in our works, probably, we are going to be replaced by those using these technologies, we also need to evolve!"

?

Luis Escalante C

Master in Artificial Intelligence