Fair Algorithmic Decisions (7/8)

This is part 7/8 of a series of posts on Fair Algorithmic Decisions. To find back all of them, please go to Posts on my LinkedIn profile. You can also contact me via LinkedIn messages to get the full document.

Manage: Understand the root cause

As said earlier, two dimensions need to be considered in careful detail before making a decision with respect to algorithmic fairness, namely the context of the decision process, and the root causes that led to the observed behaviour. We now turn to the latter. There are numerous potential approaches to determining whether an algorithm represents risks to fairness in view of the multitude of fairness definitions. The situation becomes even more unwieldy when considering the potential root causes for the issues with the algorithmic decisions, and ways to address the shortcomings. We refer to the literature for further guidance, and limit ourselves to some pointers here.

(ii) The data collection process of the training data may itself be skewed and already introduce an incomplete view of the world. Think for example of election polls that often get it wrong. The channel by which people are contacted for such surveys, but even more their propensity to answer honestly, create bias in the political opinion prevalent in the sample of answers. So a good understanding of how the data was collected, is key. This is obviously a bigger challenge with purchased data (sample selection - completeness). Similarly, the decision in the UK to use algorithms to give students A-level score during the COVID pandemic, unwittingly introduced a bias in favour or richer kids, given that schools that historically had better success rates, where also more expensive ones (sample selection – understanding reality).

To train a model for a decision process, one needs labelled data, the outcome target that the algorithm tries to predict. Sometimes historical data are objective (did someone ever get 90 days past due on his loan, or did a student obtain a certain minimum grade within x years), but sometimes these historical data labels are potentially incorporating a less accurate assessment of the underlying truth, either because it is not known with certainty (was an aborted transaction really a fraud?) or because the human assessor had a bias (surely people from that community should not be easily integrated as employees?) (data labeling). Not only does the mis-labeling issue potentially make historical unfairness persist, it also harms classifier performance relative to the true underlying classification.

Another source of breaches of fairness could be that the algorithm unwittingly found a proxy for a protected variable. I.e. some combination of the different features in the model, not obvious during a casual model analysis, is well correlated with a protected variable, and this combination de facto drives the algorithm outcome. This can lead to indirect discrimination. The more complex the model (e.g. deep learning), the harder it will be to detect that such mechanism is at work.

Furthermore, it may be that the model is trained for a wrong objective using the wrong features. The choice of the objective function, and the universe of features being considered for explaining the outcome, may reflect inherent bias in the world view of the ones building the model. Consider for example a model or scorecard that in the context of a recruitment process tries to assess whether a candidate would be a good fit to the team. What constitutes loyal team membership and what characteristics contribute to that, is likely very gender, age and culture dependent. So the model design may reflect the views of the developers, and hence the composition of the developer team. On the other hand, this is also the type of cases where one should really be questioning whether it is ethical to develop such an algorithm at all, given one is on scientifically extremely thin ice in translating such “decisions” in an algorithm. Algorithms are not value-free.

Once one has identified these root causes, one can obviously start addressing the issue. This is, of course, a delicate endeavour, both technically (how to achieve the stated fairness goal) and content wise (what is the fairness we are aiming to achieve). The same level of rigorous governance that one applies to the algorithm development should therefore be used in this part of the management process too (documentation, who signs off, …). We discuss some of the options next.

Manage: Some technical options

Having identified the root cause of the algorithmic fairness problem should give a hint as to what can be done to address it. The literature often speaks about pre-training, in-training and post -training mitigation actions. We will briefly look at them in this order.

Pre-training remediation aims to address the problems with the original data set. This can range from redesigning the sample collection process, or the target labeling process, to ensuring that all the segments are equally represented through some resampling technique or the creation of synthetic data. As said, all this should be done with the utmost care and within strict approval governance.

Careful feature vetting is an important safe guard in any procedure aiming to control the risk of algorithmic unfairness. For some applications, correlation may be good enough (e.g. marketing recommender systems), while for others it is important to understand that a feature used in an score card is causally related to the outcome one tries to measure. Correlation is necessary, but the causal understanding makes it appropriate for modelling (a formal two step test). Insights as to what constitutes a genuine causal link, can evolve over time, and require a deep understanding of the business context of the problem being modelled. This is where it becomes relevant to remain on one’s guard when it comes to proxy variables that the algorithm may have uncovered unwittingly. One powerful graphical means is plotting the Information Value of a feature relative to both the target (say default) and to the protected variable (say gender). One prefers features that are information rich on the target and information poor on the protected variable.

A complex in-training technique to address imbalance is adversarial learning. During the training, one modifies the predictor on the target as long as it is also a good predictor on the protected variable. This most readily shows that de-biasing may come at a price in terms of predictive accuracy.

Provided that the rank ordering of the algorithm is comparable between different segments, and that the problem is “merely” that score distributions are not the same over the different segments, by far the simplest approach to re-equilibrate the different segments, is through a post-training model calibration, where (for instance) a linear transformation is applied to the score, the logarithm of the odds of the positive outcome.