Artificial Intelligence Ethics - A tale about Machine Learning biases
As Artificial Intelligence (AI) becomes more prolific and powerful it is enabling us to automate a lot of systems and processes which were normally handled entirely by humans. This, in turn, means that AI ends up having to 'decide' on certain matters that need some sort of ethical framework. In the specific area of Machine Learning (ML), ethics plays a role because data can introduce certain biases, if we are to be ethical we need to be cognizant of these biases.
Cognitive and Statistical Biases in People
Humans are prone to many different forms of biases, one of the most interesting of these is cognitive biases - often studied within the field of behavioural psychology.
Artificial intelligence is about mimicking or even achieving natural intelligence through artificial methods. Perhaps one of the emergent phenomena of intelligence and learning is the development of biases.
Learning is about interpolation and extrapolation based on lived experiences and situations. We go through certain experiences and we try and predict and hypothesize how other related situations will play out by basing it on a model of the world that we have learnt over time.
The above diagram is a very crude illustration of how learning-based intelligence works. Every experience is a mixture of a circumstance (inputs) and the result of being in that circumstance (output). We have a lot of these experiences (confirmed experiences), and based on these we create our own representation (learnt model) of how things work. Using this learnt model we can now intelligently make predictions of how certain circumstances will play out (hypothesized experiences) - we extrapolate and interpolate using the learnt model as a tool.
This means that our learnt model is very dependant on the experiences we have, and in turn gets more accurate when more data is introduced. For example, if you touch a cold stove plate with your hand and it feels cold, your learnt model might be that all stove plates are cold, always. If you then later touch a hot stove plate and get burnt, your learnt model will be duly updated; Next time, you will be able to extrapolate or interpolate to know if it is safe to touch the stove plate or not.
Intelligence and learning is about interpolating and extrapolating based on known experiences
Biases, particularly cognitive biases come into play when there is some statistical unreasonableness in how we generate our learnt model based on our known or confirmed experiences. Cognitive biases cloud our judgement and thinking because we end up with wrong (at least statistically) learnt models because of how we interpret the data we have (confirmed experiences). Below are some of the common cognitive biases:
- Confirmation bias: Being more skewed towards data that confirms our current learnt model and not favouring data that doesn't confirm it. This results in only paying attention to what we already agree with, and not listening to anything that counters our thinking, further reinforcing our own beliefs (which may be wrong). This is part of the reason why people with the same views often end up grouping together, their confirmation biases feed off each other.
- Anchoring bias: Being more skewed towards a certain piece of data (normally the first piece of data you come across) and not favouring other data after that anchor. For example, if my first car trip ended in an accident I might tend to believe that cars are generally unsafe, regardless of any other examples I see or experience that suggests otherwise.
- Survivorship bias: Being more skewed towards data that passed some sort of selection process (i.e. surviving past a certain point). For example, I could believe that if I drop out of school and start my own business I am guaranteed to make it big and be a billionaire and at worst a millionaire. In this case, the selection criterion is that I am only focusing on entrepreneurs who did make it big after dropping out and not focusing on those who dropped out but struggled afterwards.
- Availability heuristic: Being more skewed towards data that is readily available in memory, this could be because of many reasons such as media reporting or having recently gone through a situation with such data. For example, if I know a lot of people who seem very healthy besides consistently eating only junk food, I might believe that junk food isn't bad even if you constantly consume it (because the limited data I have available confirms that).
There are many other statistical and cognitive biases. The main takeaway regarding biases is that they are about how we interpret data or experiences. This also affects our general model of how we perceive the world and eventually how we react and interact with the world. With this in mind, it is very crucial that we are very careful about how Artificial Intelligence interprets data through Machine Learning.
Machine Learning and Data
Machine Learning (ML) is simply a learning-based approach to Artificial Intelligence (AI). ML is learning-based in the sense that patterns and trends are learned or discovered from data that is used for training the machine (training data). This is as opposed to traditional explicit programming were a problem is solved by having already existing learnings merely being encoded into a program, then run by a machine. For example, let us consider a system that predicts property (house) value based on the distance to the nearest city, amount of rooms, size, age of the house etc. This can be built in two ways as follows:
- Build up a model using whichever tools are available (e.g. statistical analysis and industry knowledge). Once that model is known it can be encoded into some sort of program that will take in our inputs and output a predicted value. This will pretty much be some sort of function that takes our inputs and outputs a value.
- Gather up historical data of numerous examples of property value information. For each set of data, we will need to ensure that we have the input values we need (number of rooms, etc.). After making sure we have enough standardised data, we then train an algorithm to predict the value based on the various inputs. The training will be done iteratively and incrementally until the algorithm does a reasonable job predicting the property value for the example data we had collected. Once we are happy with the accuracy of the predictions, we now have a model, encoded in a machine, that can predict property value based on various inputs.
Approach 1 is using traditional explicit programming whereby we just encode a known solution or method. Approach 2 is a Machine Learning approach. This is because, in approach 2, the model was discovered or learned from the data and not explicitly encoded.
With the above in mind. It is very clear that there is a very important and intrinsic relationship between Machine Learning and data - particularly, the data that is used to train the ML algorithm. The model that is learnt via ML is a direct result of the data that is used to train it. This, in turn, means that it is very possible to have wildly different machine learning models for the same concept/problem if the data is also wildly different. The different models also imply different predictions (i.e. different solutions to the problem at hand). This is true even if the exact same machine learning algorithms are used.
Thinking back to the example of property value, depending on which data is being used, the final model might be different. So it makes it very crucial to understand the data before applying machine learning.
Understand your data before applying Machine Learning to it
If your data is skewed in one particular direction, it will proportionally skew your Machine Learning system. For example, if all your property examples were for high valued properties, the pattern your Machine Learning algorithm might learn is that all properties are high valued - thus your algorithm will be very bad at predicting lower valued properties. This is one of many biases that can be introduced based on data.
Biases in Machine Learning
It is clear that the way we as humans handle data can result in some often unnoticed biases and at the same time, Machine Learning is very dependant on the data that is used in training. For this reason, biases can similarly occur within Machine Learning. What makes biases in Machine Learning even trickier is that they are not too well known and can go easily unnoticed.
To illustrate how biases can leak into what may seem like a benevolent Machine Learning application, let us consider a hypothetical situation in a very realistic setting: Suppose you work at a bank and wish to automate the house loan approval process, modernising it from a very manual process which is prone to a lot of human error and prejudice. This could be a very great application for Machine Learning. All that is needed is to get historical data - which banks, by law would have. After that, all that is left is to pick an appropriate Machine Learning algorithm/process and run your ML training. Suppose that you are able to gather at least 50 years worth of data.
Let us assume that our housing loan task is in a South African setting. South Africa has a very young democracy with a past that is clouded in Apartheid, which in particular saw black people oppressed in many different ways, including economical exclusion. South Africa only achieved democracy in 1994 and it is not like things changed in impactful ways immediately. Assuming that we are building our housing loan machine learning system in 2019, this means we have at least half our data coming from a period shrouded in prejudice which affected a myriad of sectors including the banking sector. In the days of Apartheid, this means that it would have been much harder for a black person to get a decent house loan approved as opposed to their white counterpart.
Since Machine Learning draws trends and patterns from data, this inadvertently means that our new Machine Learning system will also learn and reanimate all the prejudices from the Apartheid era in South Africa. Based on the learnt trends and patterns, our system is likely to (when compared to white applicants) hardly approve good house loans for black people. Suddenly, we have a racist machine. This is in a way similar to the availability heuristic cognitive bias, in this case, the machine's judgement is skewed because of the availability of data (i.e. the data which we use to train the Machine Learning algorithm). If the machine was instead trained on data from an equal country (or an equal period of a country), we would have a more ethical housing loan approval system.
Let us consider a different yet also realistic example: Suppose you work for a particularly large organisation which is having a challenge in their recruitment process because of the number of applications which are received. The idea here is to build a Machine Learning system which will be used for screening applications before they can be taken through to interviews. One way to achieve this is to build a data set by looking at features of both successful and unsuccessful historical candidates. Now, to make it more realistic let us assume that - like many other big organisations - this particular company was also recently accused of preferring men over women when hiring. Again, if you blindly apply Machine Learning to the data you have collected, you will resurface whichever prejudices and biases that were involved in the original decision making. In this case, you suddenly have a sexist machine. Because of the trends in the data (which in this case is to prefer men over women), the machine will act in line with the learnings. This is again very close to the availability heuristic cognitive bias.
Interestingly, the job applicant screening system could also have biases introduced partly in a way very similar to how the survivorship bias works. If it was well known that the company preferred men, it might mean that women were less likely to even apply for a job with the company even if they would have been extremely good candidates. This means that the data used to train the machine is somewhat incomplete as it is already a filtered view.
In both the case of the housing loan approval system and the job applicant screening system, there was nothing particularly wrong with the Machine Learning algorithms. The biases were introduced as a direct result of the data that was used for training the system. There are, however, other biases which can be introduced not by the data, but by the Machine Learning algorithm itself.
An example of a bias that can be introduced by the Machine Learning algorithm is technically called overfitting. Overfitting is simply when a Machine Learning algorithm tries too hard to force a model to fit known examples. This is very similar to confirmation bias (because the machine is trying to have a model that is very skewed towards confirming current examples). There are various ways to solve overfitting, but in general, the solution involves sacrificing a model that is extremely consistent with known examples to one that is more generic and thus more likely to work for new examples which were not used for training. This is very important because there can be noise affecting certain examples, so we wouldn't want our model to predict certain known examples consistently. For example, looking back at our property valuation system, we might have an example of a small single bedroom property that should normally not be very valuable but has a high-end value. The property has a high-end value simply because a celebrity used to stay in it. In this case, we would not want our Machine Learning system learning that all small single bedroom properties are extremely valuable - that would be overfitting.
Based on the above examples, it is very apparent that biases are actually very commonplace and can occur very easily in Machine Learning applications. This could be because of the data being used but can also be because of how the machine learning algorithms are applied.
A checklist for avoiding or at least being aware of Machine Learning biases
Biases can easily creep into Machine Learning applications. We might think that we are removing human bias and prejudice by using Machine Learning. However, if the training data used for Machine Learning represents those human biases, then we are simply perpetuating the same biases.
Thinking that removing the human from the equation will remove human biases is naive, if the machine is trained on the same raw human data, the same biases will resurface
Here are some checks that you should do before creating a Machine Learning system that is likely to have a big impact:
- Fully understand your training data.
- Make sure your data is representative (have a big enough cohort of data per scenario you wish to cover). This can be achieved by running statistical analyses over your data.
- Remove any blatant outliers that will badly affect your model.
- Evaluate your learnt model. There are many ways to evaluate your model (e.g. k-fold cross-validation, or simply just applying your leant model in a sandboxed environment.
- Retrain your model as new data comes in, new information will keep the learnt model updated.
By doing the above, you are a step closer to removing (or at least being aware) biases in your Machine Learning system. In turn, you are a step closer to solving Artificial Intelligence ethics.
Palota is a company that designs and develops innovative digital products. Let's collaborate, contact us on [email protected].
https://www.dhirubhai.net/company/palota/
https://palota.co.za