Passing in the Labs, failing outside isn't what you want for your AI models

In an article, MIT explained why AI brings the near-perfect performance in the labs but fails in the real world settings. They quoted two main reasons for it - Data shift and Underspecification. I will add the third one, Undertesting.

What is data shift?

AI models are trained with a set of inputs but the real-world scenarios throw different set of inputs. A change in the input dataset will influence the output variable and the underlying relationship between input and output data. So, you are not getting the same results with completely different set of inputs. Whenever there is a change in the data distribution, data shift occurs which leads to model?performance degradation.?Data shift is a problem right from the get-go and there are other issues, such as model drift, concept drift, training-skew, etc., that decay the model along the way, justifying the disclaimer, "past performance is no guarantee for future results".

What is Underspecification?

It is the failure to specify enough details when we build an AI/ML model and so we end up with an incomplete model.?Technically speaking, it happens when certain features are omitted in the underlying representations, ending up with incomplete requirements for the model. In this case, even the same model with differing starting points could produce vastly different predictions.?Requirements completeness is very difficult to achieve in AI modeling as we expect the system to face varying situations to evolve. This leads to costly maintenance of the AI/ML model, otherwise model decay will be faster, and the product will fail more often in the real world scenarios.

What about Undertesting?

Testing AI products require unique tools, techniques, and the knowledge. Testing and training are sometimes interchangeable where we let the systems learn the 'right' and 'wrong'. If the testing does not cover the acceptable range of data shifts, minimum acceptable performance levels and the gradual/sudden decay of the models, then it is undertesting.

So, what do we do?

Below are some of my suggestions. Please add yours.

1. Improving the model through ground truth: To train or test AI/ML models, you need high-quality, high-volume ground truth dataset. Ground truth is data collected from real-world scenarios to train algorithms on contextual information such as verbal speech, natural language text, human gestures and behaviors, spatial orientation etc. Ground truth describes the validation of data by going out in the field and checking “on the ground” and provides the data that is “known” to be correct. Synthetic data that are used in the labs alone is not good enough. Training and testing the data model with ground truth makes the AI/ML logic perform better.
2. AI to test AI:?When the internals of a software system are known, developing tests is straightforward. In an AI/ML solution, the “interpretability” of the AI/ML logic is low, i.e., input/output mapping is the only known element and the mechanism for the underlying AI/ML function (for example, prediction) cannot be looked at or understood. Though traditional black-box testing helps to address the input/output mapping, when there is a lack of transparency, humans will have difficulty trusting the testing model.?There are unique AI techniques such as posterior predictive checks (PPC), genetic algorithms, neural networks, fuzzy logic, etc., that help validate the AI functionality. Testing should go beyond just input and output mapping. Testing should also ensure bias, fault tolerance and failproof. I wrote a white-paper with all the technicalities on this topic sometime back for you to refer, if interested.
3. Model monitoring for performance degradation: No model lives forever. We need to monitor the health of the data, model, interfaces, and the associated services. Measuring the model quality metric and taking actions if the model falls below acceptable standards are key. Defining standards for each of the control variables in the AI/ML model and constantly monitoring if their standard deviation is within the acceptable levels is the best way to do it. When the metric exceeds the boundaries, the model should be retrained with the new set of inputs (and assign high priority to the newer patterns) to get the model accuracy back.?It is important to align the operations, testing/training, and the design teams to react when the models reach alarming levels of performance degradation.
4. Periodic model maintenance: Periodic model retraining and retesting are absolute necessity, sometimes to unlearn the patterns of the model when they are no longer valid. Training the models with new ground truth data and preparing it for the new segments, users, and categories of use will help keep the model accuracy longer. Caring an AI product is more complex than managing the rest of the IT assets. Take it very seriously.?

Learning from one's mistakes is an effective human learning technique where the learners focus more on the topics where mistakes were made, to deepen their understanding.?AI/ML logic's learning will also follow the same technique. If your systems are not mission critical, mistakes are tolerable, else, a single mistake can lead to an irreparable loss.

A single error may be good enough for consumers to lose their trust in the AI. A diligent approach to address data shift, underspecification and undertesting can help AI systems to pass everywhere and every time.?Let us make it happen.