The Gender Diversity Crisis In Artificial Intelligence And Data Science – And How To Tackle It

One of the biggest challenges facing the tech industry today is its lack of gender diversity. Despite 51% of the population of the United States being female, the percentage of female employees at the nation’s leading technology companies averages between 20% and 40%.

In the UK the situation is even more dire, with women making up just 17% of the tech workforce.

Clearly, there are still powerful societal trends at work here. Despite decades of striving to provide equal opportunities, the science and engineering roles which dominate the industry still fail to attract female applicants for a number of reasons.

In some cases, these roles are seen as not compatible with society’s need for women to take time off to have children. Another is that women are still not as frequently encouraged to pursue an education in the STEM subjects often considered foundational to successful careers in the industry.

It's important to remember that these figures are across all roles within tech organizations – taking in marketing, HR, admin, and all support functions. When it comes specifically to tech-focused roles such as engineers, programmers and data scientists, women are often even harder to find.

At Elsevier – a world-leading publisher of technical, medical and scientific journals – women currently make up 25% of its "core tech" staff. Chief Technology Officer Dan Olley tells me that while this beats the industry average, it's nowhere near enough. 

"We're doing better than most, but we're still a long way off where I want us to be," he says.

“There’s a plethora of documented evidence showing that diverse teams lead to increased company performance.

“And we know in tech, specifically, small multi-disciplinary teams are much more effective at getting things done – all the research points to the fact that building in diversity leads to better performance.”

There’s a rather simple and obvious reason for this. Science has taught us a great deal about the working of the human brain. However, not one piece of evidence among the millions of academic studies published in Elsevier’s journals will suggest that female brains are less capable than male brains when it comes to understanding and applying science and technology.

So, who knows how many great breakthroughs or discoveries the human race has missed out on, due to society’s historical bias towards educating and employing men rather than women?

While this bias may have already tragically impacted the scientific progress of our species, there’s one particular reason that it is hugely important to rectify it now – the arrival of artificial intelligence (AI).

Smart, self-learning machines are already transforming the world in innumerable ways, and the speed of this change is only likely to increase as AI becomes increasingly widely adopted and deployed across every industry.

“There’s a lot of talk about explainability, and the ‘black box’ of AI, and ethics and bias … and its all very valid,” Olley says.

“We’re moving from just programming computers to manipulate structured data, to this concept of ‘training’ machines – and when you’re trying to build a training curriculum and testing regime for these algorithms, having people with a diverse set of views and backgrounds feeding into this training is incredibly important.”

The way the traditional lack of diversity in science and technology allowed that lack of diversity to self-perpetuate, is a good analogy for the dangers a lack of diversity could pose to the development of AI.

When it appeared that all the significant discoveries and breakthroughs were being made by men, it seemed to make sense to look to more men to make the next great discoveries. Of course, this overlooked the fact that not nearly as many women were being given the opportunity to make discoveries and breakthroughs in the first place!

When it comes to developing and training AI systems, there’s a danger of the same thing happening. Machines trained on biased data (which is more likely to happen with non-diverse teams) will give biased output, and that output will then be used to train more machines, perpetuating the bias further still.

“It’s incredibly important that we have people with different experiences and viewpoints who are able to ask ‘have you thought of this?’, ‘have you thought of that?’” Olley says. “It was important before, but now it’s critical.”

So, if it's essential to put more women into tech roles when it comes to working with AI, but women are less likely to apply, what can be done?

Olley says “If you look at tech in general, there are certain areas where we are woeful – we know gender diversity is really poor – and it gets worse as you start to look at more senior roles. But because it’s so glaring, people generally accept that it’s a problem which needs addressing.”

The first instinct may be to hire your way out of it, but merely setting out to employ more women without addressing the underlying causes of why women are less likely to apply for roles within an organization can often be a mistake.

One may be focusing too exclusively on STEM graduates, Olley says – which traditionally are less likely to be women.

"Just because someone didn't study computer science or maths, doesn't mean they don't have the potential to be a good programmer – you also have to look beyond the traditional graduate intake route if you want to encourage maximum diversity in your intake," he tells me. "If you're only going to go after STEM graduates, you've just pre-excluded a large chunk of the population that you want to go after.”

As a result, Elsevier renamed its “Tech graduate program” to “Tech associate program."

"This means we have to put a lot more effort into how we're going to assess candidates and make sure we're getting the caliber we want, but we also get to completely ignore all of our orthodoxies about who went to certain schools, or got a certain degree."

Elsevier analyzed actual data to determine where the best candidates for programming roles were coming from and found that the most successful didn’t tend to have computer science degrees, at least in the UK.

Another challenge which it found necessary to address was the low rate at which employees were returning to their roles after taking maternity leave.

“We found that this was often because people thought that their skills had deteriorated while they were away – so we are putting in a program specifically to make people feel comfortable about coming back, and make sure their skills weren’t diluted,” says Olley.

“You’ve just got to think about what demographic you’re trying to attract, and how you’re going to help them achieve what they want to achieve.”

So what specific advice would Olley give for organizations that aren’t attracting female applicants for tech and leadership roles?

“You’ve got to question everything you do in recruitment,” he tells me.

“Look at your job descriptions – are they attracting the right people? Look at your external profile, and your interview process, are they putting people off?

“It’s amazing how when you analyze some of this stuff, you find a surprising level of unconscious bias– we now do unconscious bias training for everyone that works with us in tech, and that’s partly to make people recognize how bias differs between different groups of people, and its also to help the overall culture of the firm.”

Thank you for reading my post. Here at LinkedIn and at Forbes I regularly write about management and technology trends. I have also written a new book about AI, click here for more information. To read my future posts simply join my network here or click 'Follow'. Also feel free to connect with me via Twitter, Facebook, Instagram, Slideshare or YouTube.

About Bernard Marr

Bernard Marr is an internationally best-selling author, popular keynote speaker, futurist, and a strategic business & technology advisor to governments and companies. He helps organisations improve their business performance, use data more intelligently, and understand the implications of new technologies such as artificial intelligence, big data, blockchains, and the Internet of Things.

LinkedIn has ranked Bernard as one of the world’s top 5 business influencers. He is a frequent contributor to the World Economic Forum and writes a regular column for Forbes. Every day Bernard actively engages his 1.5 million social media followers and shares content that reaches millions of readers.