The System Testing Of AI

When we test systems, we don't stop with just testing of functionality of modules, or integration testing of the modules when they interact, but we also test how the system behaves as a whole when put in real-world situations. This comes under several names - system testing, solution testing, interoperability testing, and so on.

We can draw a parallel between this and how we look an AI system as a whole. An AI system is not just the model, not just its data, but has to be taken as a whole and tested on its interactions with other modules in the system, and the users of the system. It's important that we consider how the system would behave during real-time interactions, and there could be several types of testing on that - fault-tolerance, hallucination, performance, security, and so on.

NIST has started a program called ARIA to evaluate AI systems through validation and verification. It would be interesting to check out the scope of this program, and how the testing is being organized within the program, because it would be a great example for other countries to follow suit and implement similar programs in their purview.

As a testing and quality professional, I am quite curious and interested in watching the proceedings. I would also be interested in tracking how this affects other areas like software bill of materials. Real world testing and feedback would make SBOMs implement elements that are important from a system-testing perspective. So far, to start with at least, we are only looking at the machine learning models and the data. We need to start looking at AI system parameters as well.

If you are interested in having a conversation about how to look at things from a system perspective for the AI systems for your organisation, or even to have a casual chat on the topic, feel free to setup a time with me. Glad to discuss.

We are living in interesting times, for sure!