Swayam: The STEPs Model of Prompting - I

This is first in the series of articles that I am going to write in the spirit of consolidation of my thoughts on LLMs and prompting.

Background

Before I talk about the model in the title, here's some background information.

Some months back, I had published 'The 5E Model for LLM-Assisted Pluralistic Testing':

Target Audience

This series of articles is focused on the Engineer phase. In case you have not started with LLMs yet, or you are still in Engage zone - figuring out what to do or your skepticism is preventing you from using LLMs for any practical purpose in work - this article series is NOT for you, although you can of course, choose to read it and I'd be happy about it.

It is meant for the target audience who have either crossed the Explore zone, or are in the later stages of it:

• Decently well-versed with Prompting.
• Able to differentiate between a task and a prompt
• The prompts have progressively become very intricate and complex. There is a split up of instructions between system prompt/custom GPT (or equivalent)/ Root prompt in a chat vs a given prompt as a unit of conversation with the LLM.
• Working on larger tasks, where shallow/casual prompting is not serving the need.
• Have understood the respective roles of them and the LLM in a task they are planning to solve using an LLM.
• Are at a stage of realisation that the web interface based prompting or even a CustomGPT falls short for tasks beyond simple complexity levels.

Some Tips and Ideas for Prompting

W.r.t prompting itself, I'd earlier highlighted some points in a LinkedIn post, some months back. These tips also form a basis for this post and the posts to come.

• Don't give a large problem to solve in one go. Break it down. Take assistance from the LLM to break it down, if it helps.
• By letting LLM ask clarifying questions, improve your understanding of the problem.
• Direct it towards the kind of response you will like with focused prompts. Negotiate.
• Decide on the input and output formats which work the best.
• Prefer structural representation wherever possible.
• Start from less elaborate, idea-like responses and then expand on the details.
• Ask not to criticise but critique: always ask it to evaluate what's good and what needs improvement especially in self-evaluation of its own output.
• Understand yourself in relation to the LLM during initial experiments. Create custom instruction set for various situations: e.g. when you are writing a story vs when you are coding - add them upfront as a root prompt in a chat or in system prompt (user accessible configuration).
• Experiment with multiple personas for the LLM as well as the audience.
• Extract, develop and use checklists.
• Identify when a part of the problem is coding-dominated.
• Use numbered steps and other markdown constructs to instruct the LLM follow a procedure and bring its attention to parts of the prompt.
• Practice prompting to generate prompts.
• Never cram prompts, although it is valuable to create a personal prompt database to keep a track of what works for you.
• Identify input/output mechanisms, think integration.
• Be in-charge. An LLM is a tool. You are the one who is responsible for solving the problem. So, do proper context building, never consume responses from an LLM blindly - understand the background technology so that you are not led by hype, and put them in use where they can add value.
• When the prompts increase in complexity, which they will, beyond basic experiments, think Custom-GPTs or equivalent.
• (Note for me): Prompt engineering should be about how to move major part of the prompting to auto-prompting, to maintain a state of simple prompting for the user. Think API, multi-LLMs and corresponding architecture.

This "Note to me" in the above content that I wrote months back, was the trigger for the work I discuss in this series.

With deeper experiments, prompts tend to become complex. True Prompt Engineering is about reducing the complexity of direct prompting by a user, while achieving similar or better outcomes from an LLM.

Meta-Prompting and Auto-Prompting are key concepts in scaling the use of LLMs for solving a given problem.

Consistent and structured information exchange clubbed with the capability of Tool calling are important aspects of solving larger problems with LLMs.

Integrations based on Textual Schema like JSON or Custom Markdown form the basis of opening up a lot of opportunities in bespoke frameworks.

There is a lot of Python code that backs up the information I provide. I am developing an open-source framework called Swayam - pre-alpha stage as of this writing)

I am going to deliberately tone down the coding aspect, because as much as we as testers love technical details, the technical discussion itself when it happens in its true colours deters most of the testers away from it. As an example, following is the code that loads the Prompt definition from a YAML file in the model I am talking about:

Going into the above level of details would be futile. So, I'd rather focus on the abstracted concept for everyone's benefit.

You might ask - why not LangChain? The thing is that I could have developed Swayam on top of LangChain too, but what I've learned from experience is that at early stages of learning, one should try to create their own framework. It teaches a lot of intricacies that one would otherwise miss by using a ready-made framework. Another reason is of course that I am building an abstract model that I'd need to anyways code on top of LangChain too, and that code is the largest chunk of code in Swayam.

I advocate what I myself do, and I do what I advocate.

A Learner's Path for Exploring LLMs - The First Few Steps

Creating Swayam is the Step 7 in the following LLM learning path that I shared some days back:

1. Start with a frontier model's web interface: ChatGPT/Claude. Start small.

2. Pay for it. Go deeper. Learn prompt engineering. Try different types of problems, input-output formats. Learn foundations of Transformer and Attention mechanism.

3. Experiment with Root prompting - the first prompt in the chat which sets the tone for the rest of the chat.

4. Create Custom-GPT or equivalent: Look for what's repeated in your prompting. Bring it as instructions to a custom GPT.

5. Extend Custom-GPT with code/third party service as tooling.

6. Explore at API Layer: Start with Chat API. Image uploading. Response Formats. Tool/Function Calling. Context Management.

7. Create a mini-framework of your own to do repeated stuff through reusable modules.

8. Explore LangChain/LangSmith.

9. Chunking of large content. Learn about Vector DBs and Embeddings. Semantic Search. Effective Summarisation.

10. RAG. Start small. Different RAG styles.

With this background, let's begin.

The STEPs Model of Prompting

The STEPs model is a structured approach to crafting prompts for AI systems, that I've developed through my LLM experiments.

The purpose is to guide the process from conceptualisation to action. It consists of four key stages: Story, Thought, Expression, and Prompt.

Each stage builds upon the last, ensuring that prompts sent to an LLM are rooted in thoughtful, structured reasoning, enhancing both the quality of responses and the overall interaction. The STEPS model not only makes the prompting process more efficient but also allows for more nuanced, effective conversations with AI.

Story

This stage sets the narrative context. It's about defining the overarching goal or scenario in which the LLM will operate. For example, in planning a trip, the story might be about exploring cultural and historical experiences in Berlin.

Thought

In this phase, the key considerations and queries are formulated. What are the key objectives? What does the user need to achieve? This stage focuses on identifying the important questions, such as the must-see landmarks in Berlin or efficient transport options.

Expression

The considerations from the Thought stage are articulated as clear intentions (directives). It's about expressing the desired outcome in detail. For instance, "I want an itinerary that balances historical sites and hidden gems."

Prompt

Finally, the expression is distilled into a precise and actionable prompt. This is what will be sent to the LLM system for execution (along with all the directives). For example, "Create a 7-day itinerary in Berlin that covers major landmarks, cultural events, and offbeat destinations."

A Slice of a Story: A Trip to Berlin

Following is a small slice of the story: "Plan my trip from India to Berlin", as one of the thoughts:

I'll pause my thoughts here. More on this in the next article.