Detecting disparate treatment in predictive models
There is no escaping that predictive models are imperfect and require trade-offs between accuracy and fairness. The only sensible response is to acknowledge this reality and try to find the best possible balance.
Predictive models go through multiple steps to develop, validate, evaluate, approve, deploy, and monitor them. When machine learning is used in scenarios where a subpopulation may be unjustly denied opportunities or resources due to prediction mistakes, it's critical to check, during the evaluation step, whether a relevant notion of fairness (e.g., equality of false negative or false positive rates across different groups) isn't being violated.
The link below illustrate the use of model fairness evaluation across different problems, detecting a condition in chest X-rays and predicting whether a loan applicant will repay or default: