Computer says no. Or does it?

On Sunday night, after a busy weekend, we sat on the sofa with the intention of watching unchallenging tv that didn’t need us to really think too much. After scrolling through the tv guide we came across a 1995 film starring Sandra Bullock called ‘The Net’ that seemed familiar, but that we hadn’t ever seen. As we think Sandra is a good actor we thought we’d give it a whirl. Based on the guide summary, there would at least be fun in seeing how far the tech presented as cutting edge had in fact dated. (Spoiler alert: a LOT!)

For those unfamiliar with the film, the premise is that hackers have infiltrated various systems and altered data, with a range of negative impacts, including changing Sandra’s identity. And Sandra discovers the truth about who’s behind it all, which means the bad guys are after her.

It was definitely unchallenging tv. Sandra was great, but the plot was hokey. The tech was nostalgic - all chunky monitors, disks and that jangly dial-up tone. There was the token British villain, and a highly unbelievable storyline that he couldn’t bring himself to actually kill Sandra - as he’d been contracted to - because he fancied her. And it turns out all you need to dispatch a contract killer with a gun is a clunk to the head with a fire extinguisher. And for being such a smart programmer, Sandra made some questionable decisions in trying to escape those pursuing her. But I digress.

The point of this post, and the connection to data protection, is that the film’s premise, and the success of the hackers’ actions, depended on the absolute blind faith that everyone had in the computer being right. Yes we were less savvy in 1995, but no-one questioned the computer data: it was seen as the absolute truth.?

Faced with alternative information the humans insisted the computer knew best. There were hospital nurses insisting the man who had just died was diabetic, despite Sandra (who they had assumed was his wife) telling them ‘he was not a diabetic’. There was also a great ‘computer says no’ moment where a hotel receptionist repeatedly insists Sandra checked out 3 days ago, because the computer said so. Despite her standing in front of him arguing she was in fact still there.?

We’re still debating today about whether computers know better than humans or not, and there are plenty of studies and examples to support whichever view you want to take. Given the current hot topic of generative AI and its ability to either save us or destroy us, it’s still a pertinent question. The reality, as always, is much more nuanced. Unlike The Net, whose main characters are firmly in either ‘the good guys’ or ‘the bad guys’ camp, neither humans nor computers are that clearly defined.?

Those arguing that humans know best are reminded of the famous study of parole judges, whose decisions were influenced by whether or not they had eaten lunch.?

Those arguing that computers know best need look no further than the horrendous Post Office Horizon IT scandal in the UK. The reliance on computer data over any other evidence from the humans involved ruined lives and reputations and led some to suicide. While this ongoing scandal includes cover ups of known IT issues by humans, it remains an example of an insistence on the tech being right, and an unwillingness to ask hard questions.

The key point in all this remains that you should always check the data. Regardless of whether it comes from humans or computers. Accuracy is a long-established principle of data protection law, but its importance is often under-estimated. And getting it wrong can be catastrophic, especially given how frequently data feeds other data, and decisions.??

In most cases, it’s still humans who have to make a call when faced with conflicting information. But we’re not good at identifying our own biases and assumptions, and we’re often under pressure to make a decision one way or another. So take a step back and ask questions. Don’t assume. What are the implications for a decision in either direction? What is the impact on people in each case? If you take a people-centred approach this will all be familiar stuff. Both humans and computers can be right, and they can both be wrong. But it’s people who feel the impact.?

If you work in compliance you ask a lot of questions. If you work with compliance people this can be annoying. But on this one take a leaf out of our book and don’t be afraid to ask questions. Sandra got her identity and her life back with a computer virus that magically undid all the harm. If only it were that simple in real life…??

If you want help with your data protection governance, or even just help knowing what questions to ask, get in touch.?

https://www.creativeprivacy.co.uk/?