The LLM Speed Trap and the Skills Every AI Agent Designer Needs

The past few days I've had the good fortune of spending a lot of my time sitting down with AI Agent builders talking through what they had in mind and co-creating solutions with them (obviously in OpenDialog ??).

All these conversations helped me to clarify a couple of really important qualities of working with LLMs. Firstly, they are incredibly good at trapping you in their world. Once you are into a flow of building with LLM-powered functionality it is hard to think differently. Secondly, you have to learn to escape the trap. If its a trap you are not in control LLM have to be just a tool. Finally, there is one very important skill you need to cultivate and train on to be successful. Clarity.

?? I am the co-founder and CPTO of OpenDialog AI - Our mission is to transform the way users and organizations engage with business systems and processes by automating interactions through conversation. I write about product development, Conversational AI technologies, and, more generally, the impact of AI on the way we live and work.

The trap of LLMs: instant satisfaction

The trap of LLMs is that they provide instant satisfaction. You can have an idea, quickly type down something, while telling yourself "hey I am just testing this out" and under a minute you can have something that works well.

All the tool builders use it in their marketing. Us included.

Here's the thing. It is a real trap - not just something we say for marketing purposes. It actually works. You can build an AI Agent in just a few clicks.

The next step though is the harder one. After you've built an AI Agent you want to start modifying it. You want to start adding your own flare, style and have it solve your specific use case.

Once more you can dive into the tool and start typing out your prompts and define functionality. In a few minutes you can have something working. Instant satisfaction.

In moving at such breakneck speed, however, we don't realise something very important.

There are conversations we need to have with people about what is really required. There are more ideas we could explore even if we have something working right in front of us. We moved quickly, which means we've gained time. This gives us an opportunity to slow down. We need to learn how. .

How to escape the trap: learn how to slow down

Get out of the tool. Go stand by board. Get a piece of paper and sketch out the idea first. Talk it through. Write it down and share it with colleagues. Talk to your users. Test more than one idea out. You actually can now!

We can build so many different things. The tools are there to enable us to move really quickly.

In a world where everything can happening quickly we need to learn to slow down. Make sure we are solving an actual problem and think through the consequences.

The trap of LLMs is that they accelerate everything and to escape the trap we need to learn to slow down. What is amazing is that we now actually have the time to do that!

The single most important skill: clarity in written communication

Together with mastering the art of slowing down there is another skill we need to develop very diligently which is benefits from slowing down. Clear written communication.

More often than not, when I sit down with someone to walk through a solution and we are looking at a prompt the core issue is that by reading a prompt, I am not sure I understand what we are meant to achieve.

Often I'll ask them to read it out. Slowly and loudly. Then you can ask the question - "so are you clear about what these instructions are telling you to do?" If, after reading a prompt, you do not feel confident that you would be able to follow the instructions how would an LLM do it?

The most important skill to become a proficient LLM whisperer (I still have difficulty using the word engineering here) is clarity in written communication.

An LLM whisperer would be someone who is particularly skilled at understanding and communicating with LLMs. This person can craft prompts and instructions in a way that elicits accurate, insightful, and (as appropriate) creative responses.

Here’s what might characterize an LLM whisperer:

• Prompt Expertise: They know how to phrase questions or tasks clearly and contextually, using structure, examples, and constraints to guide the model’s output.
• Understanding Model Limitations: They recognize what an LLM can and can't do, steering the conversation to avoid pitfalls like hallucination or misunderstanding.
• Iterative Refinement: They refine prompts based on the model's responses, experimenting to achieve optimal results.
• Domain Adaptation: They can adjust their approach based on the domain, whether it’s programming, writing, brainstorming, or learning.
• Creative Problem Solving: They use the LLM not just for straightforward tasks, but for innovative applications like ideation, automation, or teaching.

In essence, an LLM whisperer is a mix of a technologist, communicator, and creative thinker who knows how to maximize the potential of AI language models.

Just like a horse whisperer an LLM whisperer's key quality is moving methodically, intentionally and patiently!