Securing ChatBots and Data Architecture

Wow! I look at that title and see 3 different Architecture Towers; Security Architecture, Application Architecture, and Data Architecture. And, somehow, that seems appropriate considering all three should be integral to each other.

Over the last few days, I've been approached about a project for redesigning a ChatBot solution in a secured environment, and providing a Data Architect for a potential client. Yet, when you look at the way ChatBots work, they have to integrated with a Data Architecture and, in both situations, you have to have both with security architected into each.

Want to delve a bit deeper to understand this? Okay, let's start with ChatBots.

If you really want to think about ChatBot's, they are basically just another application that is accessing a database of information. The ChatBot's typical architecture looks something like the following:

When you look at the ChatBot architecture, there's 3 basic components of the solution; the User Interface, the LLM aspect, and the Data Sources. The User Interface is basically the user interfacing through the Web Browser, so that has to be considered from a Security Architecture point of view. The LLM Aspect is focused on the language components where it takes the input from the user, converts it to usable information, and then initiates the search in the Data Sources. That's the Application Architecture components. And then there's the Data Sources that are source of the information that is provided back to the user.

So let's talk about the architectures. When you look at the components of the solution, you are, in essence, talking about the following from a high level conceptual level:

For Data Architecture, you are typically looking at a chunked storage of documents. The up and coming database for this type of information is a Vector Database but often the LLM used by the ChatBot will store some of the information into it's own neural net. But, at the end of the day, the data architecture is the structured data that the ChatBot will be pulling it's information from. These sources might be websites, documents, databases, APIs to other applications, and so on. The Data Architecture then has to consider what the structure of the data should be, the organization of the data (for ChatBots, the organization in it's data store), and the flow of data from it's origin to where the ChatBot accesses it.

As for the Security Architecture, you need to view the ChatBot as just another application. Then in that context, you have the data in flight, data at rest considerations (ie. securing the Data Architecture), and access to the information (IAM) which, you should be aware, means being aware that the permissions tied to documents do NOT flow into the AI environment because of the act of chunking the information. And you have to consider the actual monitoring of the solution as a whole. Plus there's the application security aspects of a typically off the shelf ChatBot solution (or how a SDLC will apply to the actual coding to create the ChatBot).

So, when you look at a ChatBot solution, you need to have a Solution Architect that understands the overarching solution and how the various components work together and Supporting Architects that own the individual tower architectures to ensure that the solution works properly.

From my point of view, that makes my organization that much more valuable to my clients. We're able to provide each one of those towers. For example, I have a lady located in Las Vegas that is an expert in ChatBots. I have a number of Senior Security Architects that can secure the environment. And I have Data Architects that can provide guidance to the data aspect of the solution (of which, for ChatBot and AI solutions, needs to be seriously considered). The unique situation is that there are few Solution Architects that have an understanding of the AI / ML aspects of the solution while also having an understanding of the Security & Data aspects (hence why you have Supporting Architects - they have the expertise to each component to backfill the lack of knowledge that the Solution Architect has).

All the more reason to have access to an Architecture Firm rather than just hiring individual contract Architects.

This was a short article but this came up in my conversations with a couple of clients over the last couple of days and I thought it would make an interesting article.

Hope this helps ...

-- Neil