What is Generative AI - for the non-techie

By now, everyone has written about the basics of this topic and ChatGPT probably has answered this question a million times. Today I want to write this from the enterprise perspective. How can you safely, securely, generate content for several use cases at scale? Through this article, I am attempting to set a starting point for a businessperson to think about Generative AI.

How does Generative AI work?

Like all models, a Generative AI model, whether it is a text generation model that has seen limelight recently or a visuals generation model, is a long mathematical equation that results in the next word (or pixel) given a partial sentence. I took several liberties in the preceding sentence to simplify it. Let me expand on it a bit. By “like all models”, I mean to relate that a Generative AI model should receive the same considerations as others when evaluating it for the enterprise. Some of the questions you would ask for any model are “has legal signed off?”, “can we customize these responses?”. By “is a long mathematical equation”, I want to indicate that the researchers who originally trained the model followed the same fundamental process of algorithmically and repeatedly updating a series of numbers to calculate a value until further algorithmic repetitions showed no improvements in how close the calculated value was to the expected value. By “given a partial” sentence, I mean to say that one must prompt the Generative AI model with an input that can be a text sentence or some other piece of content. The one thing I completely left out in the first sentence of this paragraph is that most Generative AI models are of the “Transfer Learning” type. If you used ChatGPT from OpenAI directly or if you deployed Meta’s Llama 2 on your own, you are probably running the “Foundation” version of the model. This model can be then repurposed to do different variations of a task than it was originally trained for. It is no coincidence that the “GPT” of “ChatGPT” stands for “Generative Pre-trained Transformer”. The word pre-trained alludes to the idea that one can use this as a starting point for additional training. Hence, an enterprise that wants to repurpose or retrain the model must evaluate options.

Fine Tuning and Re-training Options

Fine Tuning and Re-training are two different things. Re-training refers to downloading a pre-trained model (also known as Foundation Model) and changing the Neural Network Architecture (in relatively simple ways) to obtain a significantly different output than the original model produced. This is typically used to retrain a model to do a different task than it was originally intended for. For example, a model that generates text output may be retrained instead to summarize long text into a precis. In the realm of Enterprise Generative AI, it might be unlikely to justify the use of re-training. Fine Tuning, on the other hand, is when you want to constrain the model so that it is better at your objectives within the same type of task. For example, one could take the Llama 2 model which is good at generating text (among other things) and constrain it to the vocabulary of your business domain. There are several options to Fine Tune a Generative AI model. To start with, you can generate suitable output by a practice called “Prompt Engineering”. Prompt Engineering is not the same as fine-tuning although it can be used for that purpose to some extent. In a crude sense, Prompt Engineering is the process of tricking the AI to provide what you want. For fine tuning, you must choose between several options such as “full fine tuning”, “parameter efficient fine tuning”, “constitutional fine tuning”, “rlhf fine tuning” and several others. This is an active area of research, and you can expect this to become easier as new research converges. It is quite likely that as an enterprise, you will find yourself delving into Fine Tuning so that you can use a broad Generative AI model for your specific purposes.

Data Science Environments

I think this is where Generative AI differs the most compared to other AI and Machine Learning practices. Firstly, the physical sizes required for doing anything meaningful with Generative AI is order of magnitude higher than a Classification (such as a spam filter) or Regression Model (such as a house value predictor) or even a traditional Deep Learning model (such as models that can recognize names, addresses in a body of text or can find violent content in a social network). For example, the Bloom model is upwards of 300 GB in size and its training data set is several terabytes. For another example, training the Bloomberg GPT model needed 512 NVIDIA GPUs and a hard disk offering a GB per second of throughput running over 53 days. That is not large, that is massive! (rename LLM to MLM?). This means that Enterprise Architects should think about offering these capabilities in shared services so that Data Scientists can succeed at the business objective. On similar lines, Generative AI needs large amounts of storage of “vectors”. A Vector is simply several tables that, taken together, numerically represent each word (token) in a body of text or a pixel in an image – yes, a Vector is a matrix. Your Data Science environment must offer a fast vector database where data scientists can easily find their words and pixels. There are only a handful of true Vector databases that fit this bill yet and most are software layers on top of traditional RDBMS or NoSQL databases. Finally, all of this must be secured so that viruses and worms from compromised assets on the Internet do not make way into your environment and your data is never leaked out of these new types of resources.

Human Feedback

Due to the potential for Generative AI to unwittingly produce harmful content, Enterprises must institute a process where each model must pass through an audience check. I recommend you create a focused group of your users and have them run the model through its paces for several days. After passing the focused group, release the model only to a small group of users in the field and then gradually release it to all your customers. Even when fully released, allow all your users in the field to provide instant feedback on all content that they see. Inform the user that they are chatting with an AI robot. These types of Enterprise standards around Generative AI will help everyone gain trust and may lead to better adoption by business units.

These are just a few things that come to mind when I think of Generative AI for the Enterprise. We all have spent years building our organizations, we must rise to the challenge and seize this opportunity to apply AI to our businesses.