Building GenAI Agents - The approach

I've been playing around with building GenAI agents using low code / no code tools as I believe very strongly that agents are what will make GenAI widely usable. Currently these are simple productivity hacks as I get familiar with the concept and test the limits of the current technologies.

I have 2 main tools I use for building agents - www.anyquest.ai and www.clay.com . I know there are quite a few others (CustomGPTs , copy.ai , relevanceai.com , respell.ai , floqer.com etc.).

Building GenAI agents is a new way of doing things and having savvy business users leveraging low code / no code tools requires some best practices. I have built an approach based on my experience that I wanted to share.

But before I get into that, currently there are 3 choices when it comes to agents -

1. Use a pre-built agent - E.g. agent.ai (from Dharmesh Shah ), Josh the BDR agent from relevanceai.com , use a pre-built CustomGPT etc. These are all examples where someone else has built and hosted the agent. All you need to do is use it for its desired use.
2. Use AI developers that can custom build something for you using Langchain , crew.ai , LLM APIs etc.
3. Find a low code / no code tool that you as a knowledgeable business user can use to configure your own agents. You don't need to understand how to call LLM APIs, write python code etc. These provide you with drag and drop interfaces that make this easy for you. The tools listed above are examples of low code / no code.

While there is no right or wrong approach, I find this visual on the trade offs of skill, cost and flexibility as useful to pick the right tool.

My focus is on an approach that a savvy business user can use to build an agent using a low code / no code tool.

I use a 6 step process to build an agent -

1. Map out the current manual process with a focus on what the various tasks are and who is involved.
2. Then I tag each task as an integration task or a generative task. An integration task is one that requires connecting to a system to get the data needed. A generative task is the brainpower that humans apply that we are trying to replace with an LLM (research, strategy, translation, creation etc.).
3. Then I take each research task and try to replicate it just using ChatGPT (or Claude or Gemini) or AnyQuest Assistant. The intent is to see if the LLM can augment a human with the research task. If the LLM cannot, there is no point moving forward with building the agent. One hack I use here is to ask the LLM if it can do the task. It is surprisingly self aware. The way I word the question is - compared to a human, would you be better at this task.
4. Once it has been determined that the LLM can do this better than a human, I use ChatGPT or AnyQuest Assistant to come up with the prompts. This is a key step as the agent builders are great for finalizing the flow once the prompts have been figured out but it is much easier to iterate in a conversational interface till I get the right prompts figured out.
5. Once I have my prompts figured out, the next step is to now use the low code / no code tools to automate the sub tasks in the process. We will end up with a few indepedant sub-tasks as we have not tackled the integration. So the human is facilitating the integration.
6. At this point, we have to determine if the low-code / no-code tool has the ability to integrate with the underlying systems. If it can, then we can automate the process, if not we will end up with a process that is not completely automated but semi-automated. Alternatively we can use other tools to do the integrations. This is not necessarily a bad outcome, as long as the future state is still driving gains in effectiveness and efficiency. The other reason for this is that integration is a solved problem. There are tools like Zapier / Make etc. that we can use to automate integrations.

Now that we have the theory behind us, let us apply this to 2 practical examples -

1. Summarizing a podcast - I review a lot of podcasts and sometimes I just don't have time to listen to it but if I could ask my 8 top questions and get a summary, it is a good proxy and I don't have to listen to the podcast and I can save hours a week.

2. Given a job title, job description and company name generate the possible title of a hiring manager at the company and identify a few possible profiles that fit the hiring manager - I work for a hiring agency and I know an open job at a company and if I know the hiring manager (couple possibilities) I could reach out to them with a service to find them a candidate.

In the first example (podcast summarizer), all tasks are generative with no integration. I could add an integration step, read a list of podcasts from a Google Sheet and then call the agent, but I decided to keep it simple. I am happy to manually feed it a podcast URL (you can see how the human doing the integration vs the agent works in this example).

In the second example, there is some integration required to LinkedIn as I need to take some titles and then search LinkedIn for possible matches. I could try doing this with a web search but a LinkedIn search will be more accurate.

Here is what I get when I ask ChatGPT about it's capability of doing the task.

In summary, while I am highly capable in both tasks, a human might be better at determining hiring manager titles due to their contextual knowledge, whereas I can effectively and efficiently summarize podcast transcripts and identify key themes and quotes.

When asked to elaborate on the hiring manager agent this is what it came back with -

Conclusion: A human HR professional would generally be better at this task due to their contextual knowledge and ability to interpret subtle nuances in job descriptions and company-specific titles. However, I can still provide a good starting point and handle tasks at scale more efficiently.

Right off the bat I've figured out that it does not make sense to try and build an agent for finding a hiring manager from a job title, while it makes a lot of sense to build a summarize podcast agent.

In terms of coming up with the prompt. I started off by giving it the podcast and asking it to summarize it. I then realized that the output was very generic. So I added a persona and some context. Then I asked it to list out the 8 areas it would summarize the podcast. Then I asked it to generate a summary based on those 8 themes. After these iterations, I was happy with the prompt I had.

Then I configured the agent. I used AnyQuest.ai for this as it has an Assistant functionality (basically ChatGPT + Web Search, similar to Perplexity.ai ) inbuilt (so I don't have to jump between ChatGPT and an Agent Builder) and then I can right from there switch to the Agent interface. So I stay in one tool. There are other examples where I find Clay.com 's breadth of integrations more useful.

You can see what I created here - https://www.dhirubhai.net/pulse/genai-productivity-podcast-summarizer-vikram-ekambaram-ti3me

I know everyone talks about prompt engineering being the big new role in the world of GenAI but I also see the need for Agent Builders in the future. Taking an analogy from the data world, there was the need for SQL Programmers and then there was the need for Tableau authors (savvy business users) who could do their own analysis with no code/low code tools.