How to Prompt an LLM

“Write me a tweet about the fall weather in PNW”?

Don't do this! This above is an example how NOT to prompt LLMs. The result will not be great and you’re not really using the LLM to its max. This is not a good prompt for the following reasons

Ambiguity

The prompt is vague and doesn't provide clear guidelines on the desired tone, style, or content of the tweet.

"PNW" (Pacific Northwest) is a broad region with varying weather patterns; a more specific location might yield a more accurate description.

Lack of Context

The prompt doesn’t provide context on the type of information desired, whether it should be factual, poetic, humorous, etc.

If there are particular aspects of the fall weather in the PNW that are desired in the tweet, they are not specified in the prompt.

Lack of Formatting Instructions

While Twitter has character limits, the prompt doesn’t specify any constraints which could lead to an overly long or too short response.

No Expected Structure

The prompt doesn't guide the model towards a particular structure or format for the tweet, which could result in a generic or off-topic response.

Potential Bias

Without specific instructions, the language model might default to common or stereotypical descriptions which may not accurately reflect the current or typical fall weather in the PNW.

Lack of Examples

Examples can help clarify the tone, style, or format you're looking for, which can be particularly useful if you're seeking a specific type of response.

They guide the model towards the kind of content you want. This is particularly useful in few-shot learning where previous examples inform the model about the desired format or style of the answer.

In a way, examples serve as a mini training session for the model on-the-fly, helping it understand the task better.

Writing a Better Prompt

Compose a whimsical tweet about a rainy fall afternoon in Portland, similar to these examples:

'The skies over Portland wept joyously as leaves danced in the autumn breeze. ?????'

'Raindrops in Portland are just autumn’s applause for the cozy sweater season. ?????        

This is a way better prompt you are providing guidance to the LLM and a lot more context. The examples can even help make the Tweet sound like it was written by you.

Examples and clear instructions will improve your prompts, this is usually an iterative exercise so best done in a notebook style UI. So even if you’re building a prompt for a complex backend use-case feel free to use ChatGPT or the API directly in a notebook to craft the prompt.

Multi-Step Prompting

The true power of LLMs is unlocked when you begin to use multi-step prompting where the prompt is grown organically using the LLM. This is how one can solve more complex tasks with LLMs. Multi-step prompting is often combined with function calling to create “Agents”. Eg. Take this meeting transcript and create the required Trello or Jira tickets. There are currently two popular strategies related to this.

Chain-of-Thought Prompting

This is a prompt engineering technique that aids LLMs in answering complex questions or following elaborate instructions by first generating a sequence of intermediate reasoning steps in natural language. Without additional fine-tuning, special prompts can be used to make LLMs accurately follow complex instructions and work out answers to complicated questions by reasoning step by step towards the right answers1.

ReACT (Reasoning and Acting)

Focuses on combining verbal reasoning with interactive decision-making in autonomous systems. In doing so, ReACT addresses the limitation of "chain-of-thought" reasoning, which lacks grounding in the external world.

Think step-by-step. Use functions. Do not create new functions. Stick to the defined format and function schemas.

Format:
1. Thought: Consider what to do.
2. functionName(parameters in json)
3. Result: Function result.
4. Thought: Analyze result and plan next.
Repeat steps 1-4 until nearing solution.
Finally:

Thought: Prepare the final result.        

LLMClient

The LLMClient open-source library can automatically handle ReACT, CoT as required along with function calling, error-correction, etc to help you quickly build complex agent like workflows in only a few lines of Javascript code. Lots of examples here. https://github.com/dosco/llm-client/.../examples

const prompt = new SPrompt(responseSchema, functions);
prompt.setDebug(true);

const meetingTranscript = `
Manager: Alright, team. As we discussed in the last standup, we're working on building a todo list app. Let's finalize who's taking care of what and when we expect it done. I want to start by asking, do we all understand the basics of the project?

SW1, SW2, SW3, Designer: (in unison) Yes.

Manager: Great! Let's start with the frontend. SW1, since you are 
...
`

const promptText = `
Use the below meeting notes to first find workers by name or if not provided then skill and then create tasks for these workers.

Meeting Notes:
${meetingNotes}
`;

const ai = new OpenAI(process.env.OPENAI_APIKEY);
const res = await prompt.generate(ai, promptText);
console.log(res.value());        

Open Source LLMs

Prompting open source LLMs like LLAMA2 or Mistral is very similar to hosted models like GPT except we need to keep in mind that their training data might have followed a specific format that we would also need to follow. This is usually not the case as most inference engines that you’d use to run these open source models automatically wrap your prompt in the correct format but it’s still something useful for you to know about. GPT models probably also have their own formats but since you call the OpenAI API and not talk to the model direct this is handled for you.

"""\

[INST] Hi! [/INST]

Hello! How are you?

[INST]? I'm great, thanks for asking. Could you help me with a task? [/INST]

Of course, I'd be happy to help! Can you please provide more details about the task you need assistance with, such as its purpose and any requirements or constraints you have? This will help me better understand how I can assist you. Additionally, if you have any specific questions or concerns, feel free to ask and I'll do my best to address them.

[INST] How much wood could a wood chuck chuck or something like that? [/INST]

"""        

LLMs and Zero-Shot Learning

Language Models like GPT-3 or GPT-4 are trained on vast datasets to predict the next word in a sequence given the previous words. They learn patterns, relationships, and information present in their training data, enabling them to generate coherent, relevant, and sometimes creative text based on the prompts they receive.

Zero-Shot Learning refers to a model's ability to perform tasks for which it hasn't seen any examples during training. It relies heavily on the model's pre-training and its ability to generalize from the information it has seen to new, unseen tasks. Effective prompting can significantly enhance a model’s performance, making it crucial for obtaining desired outputs, especially in a zero-shot setting where the model has no other task-specific guidance.

I like to think of the “Self-Attention” mechanism as a sort of giant spot light that you guide with prompts across the vast latent space or knowledge of the LLMs focusing it on the data thats needed to complete this task.