Fighting AI Recalcitrance

Machine Learning is specific subset of AI that allows computers to learn from data and make predictions without being explicitly programmed. This system is characterized by its ability to finding patterns in data and apply those patterns to new situations to make a successful prediction. And because it's the computer that defines the algorithm for predictions, its very important for us to know how accurate its predictions are when compared to the real world results. However maximizing the accuracy of our predictions is not always simple. Just like in the real world where we can never travel at 100% the speed of light, there are limitations in this world of computing, which prevents it from reaching its 100% accuracy no matter how much effort we, or even for that matter Steve Roger, puts in. Now comes the Big Punch, the harder Steve tries to maximize his AI model the harder it gets to achieve 100% accuracy!!! This article is about understanding these limits and how we can work with our AI infrastructure to maximize it's accuracy.

When working to improve an AI model, we should always monitor it's responsiveness to improvements in accuracy compared to the effort we put in to make the improvement. But as stated before its accuracy will never reach 100%, and what's more, it's response to our improvement efforts will also start to diminish after a certain threshold as it becomes "stubborn" resisting any further improvements. At this point any further effort to improve its accuracy will actually start diminishing it's accuracy. This is called the System's Recalcitrance.

An AI system's Recalcitrance can be defined as the resistance to improvements in model accuracy in relation to the efforts put into maximizing it's prediction accuracy.

According to this, any business seeking to maximize its operations through the application of AI & ML, should be aware of their proposed AI system's Recalcitrance curve and work within it's limits without going overboard. If not they risk doing more harm to their business with the implementation of an overfitted AI model than without the implementation of any AI solution.

To illustrate this I recently built a simple AI facial recognition model using FaceNet with the objective of recognizing my family members from a collection of holiday pictures. I trained this model on 100 photographs and set aside 20 more for validation. Obviously the accuracy for this AI model was determined by how accurately it identified and labelled my family by testing the model predictions on a Validation set. I was after maximizing the Validation Accuracy by altering a hyper parameter called Face Recognition Threshold (FRT) that ranges between 0 and 1. I managed to increase the Validation Accuracy to 78.6% by increasing the FRT till 0.7 after which its accuracy started dropping. I had overfit the model and it was no longer performing at it's peak level.

Maximizing a system's accuracy through it's Recalcitrance Curve is one thing, but what if a business aims for an accuracy that is greater than what is currently achieved? Let's say I am into diagnosing medical images to predict medical anomalies where my earlier accuracy of 78.6% may not suffice, is there anything that can be done to increase our accuracy? Luckily there is, and there are 3 ways to increase an AI system's accuracy.

An AI system's Accuracy can be increased by controlling 3 critical system components, Input Data, Computing Infrastructure and Computing Structure.

1) Increasing AI Accuracy by Improving Training & Validation Data:

This is the easiest way to improve a model's accuracy. If I go back to my facial recognition model, this simply means providing the model with more training images. Instead of the 100 images I initially supplied for the training set, I may consider supplying 200 or may be 500 images. Or I could also consider improving the quality of Training Data by providing the model with better images that has examples covering images taken from different camera angles, lighting conditions, locations etc. This would enable the model to know how each face would look in a variety of scenarios and thus strengthen its model parameters.

2) Increasing AI Accuracy by Improving Computing Infrastructure:

This is often clubbed with the previous step where with an increased data load the need for more capable computer infrastructure becomes evident. This is also the most expensive of the 3 options as computing resources don't come cheap. In my example of facial recognition I could consider building my model on more capable GPUs that can handle multi million model parameters with ease. I can also increase its storage capacity so it can store more training data and work with higher resolution images. It could also take the form of hiring a bigger workforce to build and maintain the AI model. Using this technique we would be far more capable to build bigger and better models that scales up the performance in a way similar to how a Sports Car compares to a Small Car in speed.

3) Increasing AI Accuracy by Improving the System Structure

This is probably the most effective of the three but also sometimes the most difficult and time consuming one too. A Structure could be anything that defines how the ML model works to generate the output, starting from the Algorithms to the complete computational Architecture itself. For my Face recognition use-case, I could improve my accuracy by just make a simple switch from say using a MTCNN algorithm that is good in "detecting" faces to using a FaceNet algorithm that is more advanced and specializes in face "identification". Or I could build a completely new computing architecture. A classic example of building a new computing architecture was when the incumbent RNN architecture was superseded by the Transformer Architecture to increase the next word prediction accuracy in LLMs, giving rise to the present day boom in AI multi modal capability.

While all these 3 techniques have their inherent advantages and disadvantage using all three in combination will provide the best uplift in any AI system's accuracy. However, just as it's impossible for anything to at travel 100% the speed of light, an AI system too can never achieve 100% accuracy. And as Business Leaders considering AI as a solution, we need to keep in mind the Diminishing Returns we will encounter when allocating computing, business and financial resources into an AI system thanks to the Recalcitrance Curve. The best implementation of AI is when we pick to fight the right battles. Obsessing after Accuracy, while it's important, isn't one of them. We need to be prepared "to loose the Accuracy battle sometimes to win the Implementation war".

#AI #ML #Artificial #Intelligence #Machine #Learning #Accuracy #MTCNN #FaceNet #RNN #Transformer #Architecture #Implementation #Recalcitrance #Curve #RecalcitranceCurve #Algorithm #Data #Inputs #Outputs

PS: As an AI consultant with a focus on SMB's, I can help you explore how you can leverage the power of AI the right way to benefit your business. Whether you're looking for AI implementation, data analysis, or customer insights, I can tailor solutions to your specific needs. Feel free to leave a comment or send me a direct message to discuss your AI goals.