The data-driven approach to removing bias

Even the most mindful and aware person will still be making plenty of decisions each day that are informed by unconscious bias. And those perfectly human biases even find a way into the analytics and AI systems we create. So, can these same data systems save us from ourselves, so to speak?

To find out, I was lucky enough to speak to writer, engineer and award-winning social advocate Yassmin Abdel-Magied as well as best-selling author and Associate Professor at UCL, Hannah Fry at our QlikWorld event earlier this year to hear their thoughts on how bias affects organizations.?

We also discussed what the data say about bias, and how a data-driven approach enables us to reduce biased decision making. Here are some of the highlights from our discussion:

Paul: It seems to be almost an innate feature of human decision making that we trust our gut. We jump to conclusions, we make snap decisions, and yet we know that those are sometimes unconscious.? So how can data and analytics help us uncover when we are being biased in our decision making, either in professional decision making or more broadly in our communities???

Yassmin: It's a reflection of the fact that our brains get 11 million pieces of information at any given point, but can only process about 40 at a time. It needs to make shortcuts to be able to handle all of that information.?

And some of the shortcuts are useful in that we see the color red and we think of danger. But some of those shortcuts are actually based on faulty information, or it means that we make decisions based on an impulse or gut instinct. Right at a stage where we should be thinking a little bit slowly, a little bit rationally, we're making a decision.?

We can have confirmation bias where we take new information and we somehow reshape it. We mold it to reinforce our preexisting beliefs. Or you have anchoring bias with the first bit of information that you get about something. And the decision that you make about that first bit of information then informs all the other ways that you deal with new information. And so there are over 100 types of these cognitive biases.?

Hannah: I would encourage people to just pause for a moment and forget about data and technology, forget about A.I. and machine learning for a moment, and try to think of one single system in the entirety of the history of mankind that was unbiased and perfectly fair.

I've spent quite a lot of time thinking about this, and I don't think it is possible. So the conversations that we have around it shouldn't be about removing bias. They should be about acknowledging that bias is going to be there and recognizing which biases are dominant.?

There was a story that broke in ProPublica about the unfairness and the racial bias in an algorithm that predicts whether or not a defendant will go on to commit another crime in future. This particular story really exposed this idea that these algorithms were not making the same kind of mistakes for different defendants of different ethnicities.

Actually eradicating issues around fairness and eradicating bias completely are not going to be possible when we live in a biased world, and our technology is really a mirror of the existing inequalities that we have.

Some definitions of fairness are mathematically incompatible with others, and a perfect bias-free system is impossible. If all of us have biases, the question is what are the ones that we want in our system? What are the biases that we think are damaging or working against what we want to achieve???

Paul: How important do you think it is that we actually teach data literacy before people get into the workforce in school or through college, or in the workforce itself???

Yassmin: All of us have the ability to apply the tools of data analysis to the information that we have to make informed decisions. We often talk about getting more girls into STEM science and math subjects. The earlier that we introduce to people, the better. But also the more often that we reinforce that this is not an exclusive thing, that this is something that all of us should be part of, better decisions will be made, leading to better business outcomes and better outcomes for society.

Hannah: I think that mystique just actually ends up being really problematic when it comes to implementing these things into the real world. I totally agree with starting people young with these ideas and not pretending that they are something that is only for the Masters or the computer science nerds.

Bad algorithms have been used and people have been too shy or put off for interrogating it because they feel like it's something they won't understand. There was an algorithm that would decide how much certain disabled residents would get in state benefits. And it was coming up with all these numbers that seemed like they were plucking them at random.?

This group of people got together to see what's going on, and they were told that it was a very sophisticated algorithm that they weren't allowed to touch. In the end, they had to file a class action lawsuit to get this thing overturned. And it was just an Excel spreadsheet. I think that's why we end up getting in these sticky situations where we feel like algorithms are sort of imposing their will against us.?

Paul: With the interaction between humans and machines being the way work is done in the future, how do we ensure we aren’t instilling bad habits or biases to really make the most of the benefits that a human mind and an artificial mind bring to problem solving?

Hannah: I mean, that is it's so tough when you take something that doesn't offer any human way and context, doesn't understand you, and then you just drop it and expect it to work. You have to start with the human and design your technology around them rather than the other way around.?

With driverless cars, humans are very bad at paying attention and we're not very good at performing under pressure. So some versions of driverless cars essentially expect the technology to do all of the grunt work and the humans just sit back. And then just at the very moment when something goes wrong, the human? is expected to step in and save everything, perform at their absolute best with no notice whatsoever. And that is just not a situation that is ever going to end well.?

I really think that it's about noticing that humans and machines are flawed in fundamentally different ways. And you have to think about how those things match up and align in order to create the best possible partnership between the two.?

Yassmin: We are in a time when we are creating the foundation for technologies, an d we are still figuring out what's okay and what's not OK. And that's fantastic but also scary because if we make the wrong decisions, as we've seen over the last few years, that can have huge impacts on people's lives.?

If you’d like to hear the full discussion, including some incredible anecdotes and ideas from Hannah and Yassmin, I encourage you to view this and all of the sessions from this year’s event at: QlikWorld Online 2021 - On-Demand.

About the author:

As the Global Head of Data Literacy at Qlik, I lead an integrated, business-driven approach to enabling companies to become more data-driven. I’ve spent decades developing advanced data and analytics solutions for Fortune 100 companies, and I’m a longtime advocate for business-driven data strategies and best practices. If you’d like to learn more about how leading companies are transforming through data literacy, please visit https://www.qlik.com/us/bi/data-literacy.