Optimizing AI Outcomes through Expert Guidance: The Expert-in-the-Loop Approach

Lately you can’t attend an industry event without broaching the topic of AI. Over the summer, I was at a private Upwork event in Boston at local favorite Moo. It didn’t take long before intros turned into conversations about LLM models, machine learning and what constitutes AI and what didn’t. After that the conversations shifted to various generative AI tools and our experiences, the term “Human in the Loop” (HITL) popped up, we all agreed on one thing, you can not just put any human in the loop, you have to put someone who understands “what good is”.

Before we explore that let’s discuss HITL. HITL is a paradigm in which a human is involved in the decision-making process of an artificial intelligence (AI) system. This can involve tasks such as data labeling, model training, and evaluating model performance. HITL is often used to improve the accuracy and reliability of AI systems, as well as to ensure that AI systems are aligned with human values and ethics.

Here are some key characteristics of HITL:

• Human expertise is used to guide and refine AI outcomes.
• The human in the loop has a deep understanding of what constitutes "good" in the context of the specific AI application.
• The human in the loop is responsible for setting the bar high for AI performance.
• HITL is crucial for optimizing AI outcomes and ensuring that AI systems are used responsibly.

So let’s explore a real world example of HITL. As a seasoned designer and former Creative Director with over a decade of experience, I have spent countless hours scrutinizing thousands of photographs to achieve the perfect look for ad campaigns and designs. Through this extensive practice, I have trained my eye to discern what constitutes high-quality photography and the right look for the project. Recently, I experimented with an AI app that generated images based on my inputted text. While the AI tool was impressive in its ability to create visually appealing graphics that would almost be impossible in a live photo shoot, I still needed to invest a day meticulously combing through the results, refining my input data to select the perfect or near-perfect photo. Here is a quick overview of my first AI generated photo versus my last.

First Photo

Final Photo

This experience underscores the significance of human expertise in guiding and refining AI outcomes. As Malcolm Gladwell observes, it takes approximately 1000 hours of dedicated practice to gain a competitive edge in a particular field. Similarly, I believe that to become an expert in the loop (EITL), one requires a comparable investment of time and effort. Developing a deep understanding of what constitutes "good" if not “great.”

Moreover, if you lack a refined understanding of what constitutes quality or if you haven't had sufficient experience to set the bar high, it may be challenging to obtain exceptional results from AI. Relying solely on AI without expert human intervention may result in mediocre outcomes.

In conclusion, the expert-in-the-loop approach is crucial for optimizing AI outcomes. By leveraging the knowledge and expertise of individuals who set a high bar of excellence organizations can harness the full potential of AI to achieve exceptional results.

What are your thoughts? What are other practical applications of AI that you experienced that needed an "Expert-in-the-loop"? I'd love to see some your worst photo experiments with AI. Leave your thoughts or images in the comments.