Navigating the Generative AI Jungle: LLMs and the Real Magic Behind Generative AI

Remember the look on your child's face when you finally reveal that Santa Claus isn't real, and their parents place the presents under the tree on Christmas night? The blend of shock, disbelief, and a touch of disappointment?

Imagine the same reaction when you explain that large language models (LLMs), like the ones powering today's generative AI, aren't intelligent. They don't "think" or "understand" in the human sense. Instead, they operate based on sophisticated statistical calculations and pattern recognition.

Generative AI, despite its impressive capabilities, doesn't possess consciousness or genuine understanding. It predicts and generates texNavigating the Generative AI Jungle: LLMs and the Real Magic Behind Generative AIt based on the vast amounts of data it has been trained on. While it can mimic human conversation and produce creative content, it's important to remember that these outputs result from complex algorithms and probabilities, not conscious thought.

The magic lies in the incredible engineering and computational power behind these models. Just like the wonder of Santa Claus, the marvel of generative AI comes from the illusion it creates. As Arthur C. Clarke famously said, "Any sufficiently advanced technology is indistinguishable from magic".

At this point, I am generally asked, 'But then how does GPT chat work?' Here, I find myself in a bind, trying to distil the complexities of a Generative Pretrained Transformer into a simple explanation suitable for a casual chat over coffee during an office break. How do you explain that this technology, which seems almost magical, is fundamentally about processing vast amounts of text data to find patterns and make predictions?

LLM as a brain

Imagine a large language model (LLM) as a brain. Like a newborn, an LLM starts as a blank slate, full of potential without specific knowledge or skills. An LLM and a newborn brain have vast networks ready to be shaped by experience and learning.

If we ask this untrained model, "What is the colour of the sky?" We won't get any meaningful results. It's similar to asking a newborn baby the same question. Although equipped with the neural structures, the baby must have the experiences or learned knowledge to answer. Instead of a coherent response, you might receive a smile or curious expressions as the baby tries to process the unfamiliar stimulus.

This analogy highlights a crucial aspect of both artificial intelligence and human development: the necessity of training and experience.

LLM training

Training an LLM involves exposing it to extensive datasets containing text from books, articles, websites, and other sources. Through this process, the LLM adjusts the connections between its artificial neurons, learning to recognize patterns, meanings, and relationships within language.

But how can we obtain a valid answer to the question, "What is the colour of the sky?" such as "blue"? Let's follow this example to understand the process:

Imagine you take 100 individuals and ask each of them what colour the sky is. Suppose that most of them answer "blue." However, others, for different reasons, might answer "grey," and fewer might answer "red," thinking of sunsets, or "pink," thinking of sunrises. Despite the variety of responses, the most common answer is "blue."

So, when we ask, "What is the colour of the sky?" the LLM examines the text data to detect patterns in language usage. It learns that "the sky is blue" is a frequently occurring phrase and associates the colour "blue" with the sky.

LLM temperature settings

At this point, I'll tell you a secret: when playing with ChatGPT or another LLM, you can set the "temperature." This value influences the model's response to creativity:

• If set to 0, the model will always choose the most probable answer. So, when asked, "What is the colour of the sky?" the LLM will consistently respond with "blue," as this is the most common and likely answer based on its training data.
• If you set a higher temperature value, such as 1, the model will consider less probable but plausible answers. If you ask the same question multiple times, you might receive varied responses like "grey" or "red," reflecting different contexts such as overcast weather or sunsets.

Adjusting the temperature allows you to explore the model's learned knowledge range and see how it balances the most common responses and other valid possibilities. This feature demonstrates the flexibility and depth of LLMs, showing how they can simulate various human-like responses based on context and probability.

LLM fine-tuning

You will have heard of the 'fine-tuning' of a linguistic model. Just as human learning doesn't stop at childhood but continues through education and experiences, developing large language models (LLMs) extends beyond initial training. This next phase is known as fine-tuning. Fine-tuning involves taking an already trained LLM and further training it on a smaller, more specific dataset to adapt it to particular tasks or domains. The primary goal of fine-tuning is to enhance the model's performance in particular contexts.

Continuing with the analogy of a learning child and our LLM, imagine you have a child who has been taught general principles of good behaviour, such as being polite, sharing, and saying "please" and "thank you." These broad guidelines help the child navigate various social situations with a basic understanding of appropriate behaviour.

However, when this child is invited to a friend's house, they need to learn and adapt to the specific house rules of that home. Learning a new environment's particular expectations and customs is akin to fine-tuning a Large Language Model (LLM).

Scalability: Unveiling the Magician's Mysterious Tricks

Picture yourself as a spectator, having witnessed most of a magician's enigmatic illusions unravel before you. With this newly acquired knowledge and understanding, you can approach ChatGPT and ask about the sky's hue. What unfolds is not the naive response of a child at the beginning of their educational journey but rather the eloquent and well-informed reply of an individual who has grown in wisdom and comprehension.

Yet, despite grasping the intricate workings behind the curtain, the sense of enchantment remains undiminished. Every aspect retains an aura of magic, a testament to the enduring wonder that persists even as we unravel the complexities beneath the surface.

The true marvel lies in the scalability of language models. The immense size of neural networks and the vast expanse of training data culminate in unparalleled capabilities. These colossal dimensions empower them to generate responses that frequently surpass human performance, pushing the boundaries of what we once thought possible.

This scalability allows language models to tackle increasingly complex tasks and maintain their magical allure. As we continue to expand the size and scope of these models, we unlock new frontiers of understanding and expression. The magician's tricks may be unveiled, but the true magic lies in the limitless potential of scalable language models to captivate, inform, and inspire.

I leave you with one last magician's trick. Did you realize that when you ask a language model a question, it replies by streaming one word after another? This happens because the language model does not know the answer a priori but generates the most probable word, one at a time, based on the context of the question and its training data.

Conclusion

In generative AI, large language models (LLMs) are the true magicians behind the curtain, creating the illusion of intelligence through sophisticated algorithms and immense datasets. The true magic of LLMs doesn't lie in conscious thought or genuine understanding but in the remarkable engineering and computational power that animates them.

Like children, LLMs embark on a learning and growth journey. They adapt to specific contexts and domains through extensive training and fine-tuning, enhancing their ability to perform various tasks. The scalability of these models enables them to tackle increasingly complex challenges, continually expanding the boundaries of what we once believed possible.