LLM Programming

We are almost at the one year anniversary of OpenAI's introduction of ChatGPT - November 30th, 2022. There is an earlier date that is equally important - June 11, 2020 when GPT-3 was first released. That was the moment when OpenAI became the first company to release a foundation model which was actually useful. There were certainly some early researchers impressed enough about a year earlier (February 2019) when OpenAI released GPT-2 to point out that this new model had enormous promise (GPT-2 is not as dangerous as OpenAI thought it might be as a reminder of what smart people were saying just a few years ago...)

Indisputably though, the November 30th release awakened the world to the potential of this new technology. While there were improvements in the model itself (and the company acknowledged this by calling it version "3.5") the biggest change from GPT-3 was in the way users accessed the model - a conversational interface. While this made GPT technology accessible, it also framed the way to use the technology too narrowly with most people only thinking of it in this conversational mode.

Ethan Mollick wrote about this in early November, "Working with AI: Two paths to prompting" and the goal of my article will be to push that thinking a bit further. If you are looking to use LLMs in enterprise use cases, you will want to take the second path, which Ethan calls "structured prompting"

First let's summarize the difference between these two approaches to using an LLM (with thanks to Ethan Mollick's excellent longer discussion linked above). Conversational prompting and structured prompting represent two distinct approaches to interacting with AI systems:

Conversational Prompting:

• Nature: Informal, intuitive, and akin to a casual conversation with a human.
• Method: Users engage with the AI in a conversational manner, providing context and explaining their needs in a free-flowing format.
• Flexibility: Highly adaptable to varied topics and questions without requiring specific structures.
• Context Sensitivity: Effective use requires providing contextual information to guide the AI towards more relevant and specialized responses.
• Strengths: Ideal for exploring ideas, brainstorming, and receiving general information.
• Limitations: May lead to less precise or focused responses due to its open-ended nature.

Structured Prompting:

• Nature: Methodical, structured, and designed to elicit specific, task-oriented responses.
• Method: Involves crafting prompts with clear goals, step-by-step instructions, and defined roles/goals for the AI.
• Precision: More likely to produce consistent and targeted outputs, as it narrows down the AI's response scope.
• Components: Includes role/goal definition, explicit instructions, constraints, personalization, and sometimes few-shot examples.
• Strengths: Useful for specialized tasks, complex problem-solving, and when precision is critical.
• Limitations: Requires more effort and expertise to create effective prompts and may be less flexible in handling varied queries.

In essence, conversational prompting is akin to a fluid, exploratory dialogue, while structured prompting is like giving the AI a detailed blueprint to follow for a specific task. Correctly configuring a structured use case is effectively a programming task. One of the reasons people struggle to understand the applicability of generative AI to business problems is that they are trapped in thinking about only a conversational approach and they haven't considered the benefits of programming an LLM.

So how do you program an LLM? Here are several guidelines:

Be precise and unambiguous: The programming language you will use is English or French or whatever language you speak. A "natural language" -- one that evolved naturally through usage by humans. Contrast this with the programming languages we typically use which are logically designed with precise and unambiguous terms that follow strict syntax and structure rules. LLM programming benefits from using natural language in a precise and unambiguous manner.

Logical design: Writing a computer program requires a lot of planning -- what do you want to happen first, second, third... What do you do if X happens vs. Y happens. This same logical design should go into your LLM programming.

Multiple steps to accomplish a goal: Don't try to do too much all at once but rather consider how your desired outcome is a series of steps in a process. Then break up your LLM instructions into these steps where you have more control in creating a consistent output with each one. This will allow you to build your desired outcome through the series of steps, directing the LLM precisely through the process. This can require a whole series of requests, not just one, to the LLM.

Context management: Perhaps the most important topic of LLM programming is in the instructions that surround and create context for the specific direction that you are giving the LLM in a particular step. A simple example which OpenAI has built into their ChatGPT interface is called "custom instructions" (available as a setting when you are logged in). This feature lets you create an overarching context for any of your conversational prompts. "Always speak like a pirate in your answers" might be a silly context to use. But context can have many layers in a structured LLM usage and managing how these layers are used at each step is crucial to generating consistent and accurate results.

Hopefully this brief outline will give you a start on thinking about how to use LLMs in a completely different way from the conversational interaction of a ChatGPT!