ChatGPT + Low Code = ??

When interacting with ChatGPT (or similar apps), the chat "app" normally leverages an LLM (large language model) like text-davinci-003 or gpt-35-turbo. LLMs have been trained against vast quantities of public data, which makes them exceptionally great at predicting what a text should look like, given a specific user "prompt", where a prompt could be an idea for a Haiku about AI, or an instruction to represent data in JSON format.

LLMs are not great, however, at answering questions about information that exists in enterprise data stores and systems, like an organization's hiring policy, or our data as employees (our leave days, calendar items, etc.). This should not come as a surprise, as enterprise data does not belong in the public space and is normally not available as a training data source for LLMs.

WorkGPT?

Where does that leave us? Are we doomed to forever use ChatGPT to generate thank you emails and cat related jokes, but never as a source of information for enterprise data? Not necessarily! Thankfully, the markup used with the OpenAI API, allows for "priming" the model with instructions and some background information that can enrich the chat session.

In the example shown above, employee data is hard-coded into the chat session, but an enterprise chat application could potentially retrieve information during run-time, creating a chat experience that would be relevant to the enterprise space - in other words, a ChatGPT app that could answer questions about HR related details, provide sales information or anything else that is accessible only "internally" (and would still support the usual ChatGPT shenanigans, i.e. cats).

Enter: Low-Code/No-Code

Building and deploying a secure enterprise chat experience which relies on ChatGPT and has access to "internal" data, is no small feat. Luckily, this can be made considerably easier with Microsoft Power Platform. The design is pretty simple: PVA (Power Virtual Agents) can be used as a "Broker" for ChatGPT, that will orchestrate the chat session, and would be deployed in an employee facing portal or in Microsoft Teams. PVA can also leverage Power Automate to securely access enterprise systems and content (like an HR database) and have the chat session enriched with live, personalized "internal" data.

For example, the video below shows a PVA+ChatGPT bot, running in teams, answering HR related questions based on the user's personal profile. The first two questions in the chat are answered based on personalized HR data, retrieved in run-time, whereas the last interaction relies on more general ChatGPT capabilities.

How does one Low-Code this?

A solution pattern allowing PVA to act as a broker for ChatGPT is described in detail in this excellent post, but a few extra steps are needed to load data in run-time into the ChatGPT "system prompt".

After completing the steps described in Henry's post, make the following changes:

(1) In the Greeting topic, use Power Automate to load details about the user interacting with the bot or any information relevant to the chat session, and store those details in a variable called "employeeProfile". The variable should be set for bot level access. Since this Chatbot will be deployed in MS Teams, the user should be authenticated, and the username can be passed to Power Automate by referencing "bot.UserID".

In the example below, I am using hard-coded data, but Power Automate can access multiple data sources, including on-prem databases or APIs, and retrieve information that can enrich the chat session (think policies, calendar events, employment status, etc.). To do that, replace the Compose action in the image below, with an action calling a connector such as SQL, Dataverse, SharePoint or a Custom Connector. The information retrieved by Power Automate doesn't have to follow a specific structure.

(2) In bot framework composer, concatenate general instructions for your bot, with employee details retrieved by Power Automate, by accessing virtualagent.employeeProfile

Is that it?

Pretty much! Leveraging Power Virtual Agents and Power Automate, you should now be able to deliver an employee facing ChatGPT experience, enriched by live enterprise data.