Do non-player characters (NPCs) have basic data literacy?

It is October. The month of Halloween. Perfect timing to engage in something macabre and frankly, a little spooky: data literacy. What if you could chat with an AI character about a dashboard, charts, or ask curious questions about the dangers of mean... Speaking of dangers, that is exactly what I've been playing with using Inworld.ai through NPCs.

Business problem

I've built a data literacy experience with dashboards and cards for humans. It works well in an in-person setting with teams. What I needed for a virtual version was an assistant. An intelligent character that users can ask about specific data literacy, statistics, etc. Of course, chatGPT is a good one as it can spew out generic answers to questions. But when it comes to specifics for pilot data, especially a dashboard that does not exist anywhere, it didn't work.

For a session earlier this year, I used Poe. Not Edgar, he's dead. Spooky but not practical. I use https://poe.com the helpful chat. It allows us to build an app with pre-set information, and specifics for the conversation. But it was still far from what I imagined.

A whole world, not just a chat

What I was really after was a whole world rather than just a single chat. Ideally, this world has a backstory, and fundamental knowledge, and within that world would live a set of characters. Characters who are aware of the facts (like pilot data), even aware of each other.

As for the characters, I'd love them to have their own "world" as in personality, communication style, personal knowledge, etc. And if beyond that, I could also "train" them somehow to explain data literacy to users within the scope of the pilot data and dashboard I created, that would be awesome.

This is the reason I started playing with Inworld.ai . I created a world for my data literacy pilot that included Dr. Dash, a living dashboard you can ask about its data.

The user story

In my initial sketch of the interaction, I imagined the following user story. The user enters their name, so Dr. Dash can talk to them.

Then Dr. Dash would introduce themselves as a living dashboard, if possible even displaying the dashboard itself.

From there on, the user can type in any question, comment, request, etc. My front-end app would send this message to Dr. Dash, who would then reply. Then, a message would show up in the chat.

The screenshot shows how Dr. Dash is aware of specific facts about the pilot and uses them in her response (261 sales folks, 85 or above average score, etc.). These are world facts that she relies on in her conversations. Nothing here is set verbatim.

What about images?

The last thing to figure out was how to show images in the chat that are not part of the conversation and yet relevant (such as a chart) since I won't have control over the conversation. For example, if the user specifically asks for chart 2.

As I was building out the character of Dr. Dash, I also realized I do want to take control over some of the answers, while letting her improvise for others. For example, when the user asks about Likert scale or ratio vs. interval data classification types, I wanted Dr. Dash to use a certain analogy without scripting it.

"When user asks about Likert scale" or "ratio data" or "interval data" explain the difference and use the following analogy: blahblahblah...

How to build an interaction like that?

Inworld.ai is used to create NPCs for games. They have integration kits for Unity, Unreal, Minecraft, etc. But it also has a simple API interface for the web, which is what I used to build the front end.

Here are the reasons why I selected Inworld.ai for the project:

1. The World: a workspace represents the World. In a workspace, you can have multiple characters. They will share common knowledge about the world but they will have their own personality, personal knowledge, flaws, goals, etc.

Why two Dr. Dashes? I made a copy of the original to do A/B testing.

1. The World can include specific common knowledge. For example, I set up a set of facts for the Medieval Land where the pilot took place, and then facts about each chart on the dashboard.

Inside the Medieval Land, I described the pilot process.

This information then can be "known" by any of the characters in the world. AI will use these to rely on when asked about specifics without me dictating when and how to use this information.

1. Characters can have their own basic and advanced information. Basic includes a description for the character, motivations, flaws...

What makes this tool really powerful is the goals and triggers feature. Each character can have a set of goals and triggers that shape how they are reacting and communicating.

When users ask about Likert scale or the fact that 4.4 satisfaction is not twice as satisficed as 2.2, this is the instruction:

Use the temperature measured in F analogy to explain how 80F is NOT twice as warm as 40F, explain data classification of nominal and ratio numbers, explain how Likert scale numbers should not be treated as ratio, at a minimum, we should not make decisions solely based on the satisfaction average.        

The front end of the chat includes JavaScript I wrote to communicate with Inworld through the API. If you really want to go pro, you can use Unity to build a game where NPCs are controlled by Inworld.

But... You don't actually need any programming if you're fine with the built-in application. It even has a voice, so you can chat with your character:

https://inworld.ai/arcade/er7qBsclZX6FMWHJ

What's next?

Based on the interactions users had with Dr. Chat, there's still work to do. She needs to calm down a little. More to come on that.

P.S. chatGPT (along with Bing) now can see images, including charts. I did a quick test run but for now, it is more macabre than Halloween (somehow managed to connect my chart with learning styles...).