Bias is now expanding its reign from humans to machines - The AI Bias

Other than the COVID pandemic, the one word that we have heard the most in the last couple of months is bias. We have biases embedded in our society, regions, and countries extending to bias in datasets and algorithms. We will talk about bias in AI today. Before doing some research I asked myself, what could be the result of AI bias? The maximum I could think of was the face detection software used by companies and government. But bias is not limited to that only. It has a larger span and multiple forms. First I will talk about how biases are built unknowingly in the machine learning systems and then I will talk about how these inbuilt biases in the trained algorithms affect us in multiple forms giving some examples.

How bias is built in a machine learning system?

More than 98% of AI applications that exist and are commercialized today are using machine learning and machine learning needs data to train them. The machines slowly start behaving the same as the data fed to them. What if the data itself is non-representative? What if the data itself is corrupt intentionally or unknowingly? If the data contains more and more one type of representation, then the algorithm will detect that kind more precisely than others. If one race/culture/region is more prominent in one activity and data related to those are used to train the algorithm, then a normal person can also be tagged as having an interest in that activity even if he/she does not. These activities can be from positive to very negative. These biases exist in both, the computer vision and the natural language processing systems.

Some examples to identify AI bias around us

Let me throw some examples, how biases are affecting our daily life.

The surveillance problem

Recently it was in the news that a U.S. Based pharmacy chain, Rite Aid, installed face recognition systems in their stores to predict the chances of shoplifting. Cameras in security guards' phones and the ones installed in the stores were flagging those people who were misbehaving so that they can be tracked for next time. The problem was, most of these installations were mainly in the minority and Asian population areas. So most of the data collected was from that community only. As a result, there were multiple cases when a normal person from the same community was flagged as skeptical by the system. This raised alarms and the system was taken down by Rite-aid. Several other retailers such as Home Depot and Lowe’s have faced class-action lawsuits due to which they had to remove the anti-shoplifting surveillance systems. There is no doubt that face recognition seems to be a powerful tool to prevent shoplifting but there is much work that needs to be done to build the accuracy, reliability, and fairness in the system before implementing it on the commercial scale.

AI sending people to Jail

In the US, one in every thirty-eight adult individuals was in some kind of correctional path or program by law enforcement agencies and hence the US has more prisoners than any other country in the world. Many law enforcement agencies in the US use a facial recognition system to assign scores to someone to judge the risk. Researchers and civil activists have always protested against this, saying, these algorithms particularity targets dark-skinned individuals.

This system is also used to predict future behavior, to allocate law enforcement resources in an efficient manner. The problem is that the algorithm mostly targets the low income and minority community and assigns them a high score amplifying the already existing bias and in turn, generating even more biased data by which the system is trained again perpetually.

In one way it reduces bias since Judges now make decisions based on data instead of their gut but this cannot be an excuse for providing more sentences to a low-risk individual.

There is something called race recognition … Ummm!

Marketers have been using this race recognition system to target their potential customers. Instead of individually identifying the person which might violate some PII law now or in the future, these algorithms just try to identify the race of their store visitors through a trained algorithm and associate them with the items they pick. This really helps them targeting the ads and organizing their offline or online stores depending upon height, location, and time of the year. Many face detection companies such as Face++, Cognitec Systems, and Facewatch already provide solutions to detect race, gender, age, etc. However, the problem can arise when these retailers start charging different amounts depending upon races and start offering and suggesting different quality levels of items depending upon ethnicity.

Machine learning can be a very effective tool in deciding on market segments but the same thing can be thought of as invading privacy and can be misused by authorities. On the other hand, this can also be used in a positive manner to detect racial bias in law enforcement systems and their decisions.

AI helps or hinders recruitment

AI in recruitment is available in two ways. First, asking a few questions to decide the most suitable career path and progression and second is using AI to scan resumes and judge the ability of a person. In the first case, many times the system makes a wrong judgment, for example in the case of Lu Chen who applied for Unilever for a medical position and after a 30 min test, the system suggested her to go for banking. In the second case, if the student is not good at putting keywords in the resume but really good at what he/she does, the system can still flag the candidate as inappropriate. The second case might be helpful for companies in sorting and selecting from a large pile of the resumes since the number of false negatives are very low but at the same time, companies should also build solutions for the job hunters to tackle this challenge of AI systems scanning their resumes!

Gender Bias

The bias is neither limited to color and race nor limited to computer vision systems. As per the Harward Business Review, there are many incidences when AI has adopted gender bias. It gives examples of natural language processing systems such as Amazon’s Alexa Apple’s Siri voice assistants. When they were asked the association words, and the provided the prompt as 'man' to 'doctor' and asked 'woman' to '? (what)', they came up with 'woman' to 'nurse'. The challenge here is that now in modern society these positions are dramatically changing however the data which we have is form the past. So coming up with better data is not the solution, the algorithm itself will have to be tweaked to remove the bias from the system.

What we can do about it?

As per the human-centered design thinking principle, to regulate the machine learning models, the first step has to start with the modelers – a human. When creating these models, there modelers and designers should try to avoid complexity so that the reason for the bias in algorithms can be traced. Also, they should always consider the final goal of the models to make people's life easier, provide value to the community, and improve our overall quality of life, instead of only focusing on the Machine Learning metric like alpha and mean squared error.

Government regulations will play a vital role in regulating and extracting the most out of this innovation and cutting all the negative repercussions. It should not at all hinders the AI development since the development is still in its very infant stage. Ultimately, it is our responsibility to keep a check on it so that we can handover a great start to our next generation and not let it take a dragon form in coming future.

o o o