Bias Isn’t a “Given” in AI (5/5)

Thinking about the measurement of AI/ML in terms of chihuahuas and muffins, it turns out, is pretty easy and easy to remember. Efficiency and effectiveness described in numbers enable comparison. How does the challenger system compare to the system in use? One might say the existing system identifies 10 out of 20 criminals for every 100 reviewed, or 50% effectiveness and 10% efficiency.?

Existing System

05/20 = 25% effective

10/100 = 10% efficient

AI/ML Challenger System

10/20 = 50% effective?

18/50 = 36% efficient

Based upon accuracy alone, go with the challenger. Easy decision. But how do we know if the new AI/ML-based system reflects some bias that we cannot accept? Using the same analogy, what if all the chihuahuas found were dark-haired? Or old??

With all of the media attention given to bias in AI, a lot of people assume bias is a “given”. It’s one of the most frequently cited arguments by those who fear AI technology. But bias doesn’t have to be a standard part of all AI/ML output; it all goes back to the data.?

How do you check for bias? Here are some questions to start the dialogue: What are the age, gender, and color of the chihuahuas found??

?If the algorithm found 50 chihuahuas and missed 50 out of 100 possible chihuahuas, and the 50 it found were all dark-haired chihuahuas, then the output has a bias. This again seems like a role for humans – reviewing the output.?

Because the algorithm is the data, you will find that your training data were imbalanced with an over-representation of dark-haired chihuahuas. In the AI/ML world, the bias is not a directly human bias coming through the process, but a bias in the training data. However, it’s important to note that the training data may have been generated by biased human processes in the past. Biased algorithms come from the training data. Biased training data comes from biased humans.

The reason that Amazon’s doomed hiring algorithm was biased toward hiring men was because most of Amazon’s employees were men when the algorithm was built. The reason a criminal justice program incorrectly identified Black defendants as higher risk for recidivism and incorrectly identified white defendants as lower risk was not because the algorithm miscomputed. It was because the algorithm properly learned on training reflecting biases in the underlying criminal justice system.?

Both of these examples show how good intentions and good math but bad training data can produce inappropriately biased, but accurate, algorithms. The lesson is that measuring accuracy alone is not enough.?

It is important to note that incumbent systems have plenty of biases, so choosing to not deploy innovation likely reduces accuracy but does not reduce bias. Eliminating bias in training data eliminate bias while capturing the accuracy advantages of AI/ML. But this requires human involvement.??

Measuring accuracy is easy:? use Test Data, draw a threshold, and using True Positives, True Negatives, False Positives, and False Negatives, calculate Efficiency and Effectiveness. Recognizing bias and making corrections, however, requires humans.?

Create an interdisciplinary group from across your organization, and have a discussion. Ask people with different perspectives if results appear biased. If someone identifies an unexpected number of older chihuahuas or long hair, etc., then you’ve identified the bias. The cause will lie in the training data. The standard for innovation should not be perfection, but improvement. AI/ML systems can empower front line workers to discover threats such as human and drug trafficking, without violating the security, liberty, and privacy of others. It’s as easy as chihuahuas and muffins.

This is the second article of a 5-part series. See my video short or Read Article 4 here.