The Process of Shifting from Knowing to Doing

Based on my own experience, I think AI has reached a phase where it excels at retrieving information and answering complex questions accurately. We have moved light years compared to where we were just two calendar years ago. Today’s chatbots can process massive amounts of data in various languages and often provide the right answers to data or knowledge questions.

However, as always, the quality of your input—your prompts—and the underlying data determine the outcomes. Poor data leads to poor results, but that’s not the bot’s fault. Organizations need to prioritize cleaning and structuring their data, which is essential even without digital assistants. With AI, though, bad data issues are simply amplified.

Let’s now assume that in the next few years, AI and documentation improve enough to solve the problem of knowledge retrieval entirely. What happens next? As Karl Popper* poetically put it: solving one problem leads to a new one.

Once we’ve tackled the issue of knowing, the next challenge becomes doing.

The Gap Between Knowing and Doing

We all know there’s a significant difference between knowing something and being able to do it. For instance, I can watch a few videos on dishwasher repair and explain the process in detail—but successfully fixing it? That’s a whole other matter. AI faces the same challenge. To perform tasks effectively, AI must not only know how to do something but also execute it. It needs to walk, instead of reading how to walk.

Take robotics, for example. Atlas by Boston Dynamics demonstrates how AI can learn from its mistakes to perform physical tasks (now https://www.youtube.com/watch?v=-e1_QhJ1EhQ, vs a few years ago: (https://www.youtube.com/watch?v=aX7KypGlitg).

Similar to their physical cousins, AI Assistants or AI Agents will be increasingly able to take action. but when working with non-physical, abstract materials like policies, strategies, or documentation, the challenges multiply. These require interpretation and judgment in specific organizational contexts, making doing much harder for AI agents as well as for humans to verify whether it 'failed' or not.

How Will AI Learn to Do?

So, for AI agents to take meaningful action within an organization, it must first understand how to act. Some might say, “Just ask the AI agent/chatbot; it’ll figure it out.” While it may give some good or convincing answers, there is a problem with that:

• Do you understand what the AI just did?
• Are the outcomes consistent and reliable?
• Will the AI use the same methodology each time?
• Can you replicate and explain the process if needed?

The answer is probably “no,” “no,” “no,” and “maybe. That's not good enough. Just as we expect calculators to be reliable and their outputs verifiable, we should demand the same of AI.

Models like OpenAI o1-preview show promise with their "chain of thought" approach, where the AI plans steps before executing them. However, even this depends on the quality of the input data. Garbage instructions, Garbage actions —a problem that is probably worse than the traditional "garbage in, garbage out."

Teaching AI to Work: The Process Challenge

This means the tables have turned again. Just as companies carefully work to get quality data & knowledge fed into AI, they must now focus on feeding it clear instructions on how to perform tasks.

This does raise another important question for organizations: should they rely on off-the-shelf processes or develop tailored, internal ones? Both approaches will be able to use the organization’s data, but the choice of process significantly impacts outcomes. In the future, I predict a rise in world-class, ready-made processes, embedded in AI, that companies can adopt. However, someone must first build these processes and they should be world class.

But, coming back to how we teach AI: Can you explain your processes clearly enough for others—whether human or AI—to understand? And, if you ask 3 different people in your organization will they all come up with the same process, or will you end up with 4 different versions?

The Complexity of Real-World Processes

Take job leveling as an example. On the surface, it seems straightforward: you have a job description, you level it, and you’re done. In reality however, the process often involves implicit data as well as implicit steps - in other words people do things - and use information - that is not necessarily documented, such as:

• How to apply the leveling criteria outlined in the levelling document
• Additional dimensions (e.g. revenue) that is not in the provided job description
• Consideration of organizational context (e.g., peers, managers, and reports)
• Adjustments based on experience (e.g., knowing that one manager that tends to inflate, or downplay, job requirements by picking different words)

Now, imagine also handing all this implicit knowledge to an AI assistant. If your leveling guide still doesn’t explicitly saw HOW this knowledge should be taken into account, the AI’s output is still unlikely to meet expectations. This highlights a critical point: having accurate data isn’t enough. Clear, actionable processes are essential.

Integrating Processes into AI Systems

Let’s now assume you also solved this problem. You have documented all our key processes that you want AI to action. Then, to follow on Karl Popper's thought….. congratulations: you have a new problem: how to integrate them into AI systems. Questions like these come up:

• Should the process be part of the system prompt?
• Should it reside in a vector database or embedded within the policies?
• Should it form part of a custom GPT?
• Could it be configured in a Microsoft Azure workflow or another external application?
• How to measure whether AI followed the process, or not?
• Or will Anthropic just come with another invention that solved this entirely?

I’ve been experimenting with some these approaches, and while the system prompt and workflow configurations show best outcomes so far, they are labor-intensive and not easily scalable (for the 1000s of processes an average human executes monthly). That said, I’m not too bothered as I'm sure technology will evolve to address these challenges faster than we expect.

Quality Data & Quality Processes

While AI continues to improve, organizations should improve in defining and refining their processes. This includes deciding which processes to keep in-house and which ones to buy off-the-shelf. But regardless, in the future, quality data alone won’t suffice. Clear, well-defined processes will be just as critical to ensure AI delivers meaningful, reliable outcomes.

So, the shift from knowing to doing starts with processes. Are you ready to define them?

*Karl Popper: https://www.dhirubhai.net/posts/martinsmit_for-the-science-fans-among-you-a-quote-activity-7249717988872851457-ABOs?utm_source=share&utm_medium=member_desktop

Note: The statements, views, or opinions expressed in this articles represent my own views.