The Dark Side of AI: Hidden Biases in Algorithms

In 2010, a controversial video surfaced online, claiming that Microsoft’s newest hit product, the Kinect controller for Xbox, failed to recognize people with dark skin. A similarly awkward incident was reported in 2020 by PhD student Colin Madland, who found that Zoom and Twitter's virtual backgrounds incorrectly obscured the face of a Black colleague. But discriminatory behavior isn't limited to facial or motion recognition algorithms. In 2017, Facebook's ad-targeting algorithm was found to discriminate against women for certain job ads. In 2018, a healthcare allocation algorithm in Arkansas was discovered to be flawed. And in 2020, a Black man in Detroit was wrongfully arrested after a law enforcement facial recognition system misidentified him as a criminal. In 2021, OpenAI's GPT-3 model was revealed to provide biased responses against Muslims.

The list of such incidents is nearly endless, as algorithms and automated decision-making systems increasingly permeate our lives. Algorithms determine whether we are suitable for a job, creditworthy, suffering from a particular illness, or even what we should read, watch, buy, or eat. You may have encountered posts on Facebook complaining that the algorithm selects the wrong friends to appear in your feed and encouraging you to share a post to “outsmart” the system. While there’s some truth to the claim that the algorithm filters based on activity, the solution is far from simple.

Since most people get their information exclusively through social media, it’s no exaggeration to say that social media also shapes what we perceive as reality. Escaping this manufactured reality is extremely difficult, as the algorithms around us continuously reinforce the labels they use to define us.

Hidden Biases in Algorithms – The Dark Side of AI

“Algorithms can minimize the subjective errors of human judgment,” claim many technology enthusiasts. However, this optimism ignores an important fact: the mathematical models driving today’s data economy are created by humans, and humans can make mistakes. Moreover, these models learn from historical data, which often carry hidden biases. Algorithms then use these patterns for behavioral predictions.

In HR, for example, candidate screening systems often operate on this principle: the characteristics of successful individuals in a given position—such as their education, personality, or style—define success and inform screening criteria. If men dominate a particular role, the definition of success can become discriminatory through such proxies. A prime example is Amazon's recruitment algorithm used between 2014 and 2018, which was biased against women because the training dataset was overrepresented by men. This pattern was perpetuated, excluding many talented female candidates.

Derek Mobley sued Workday Inc., alleging that its AI-based hiring system discriminated against him based on race, age, and disability. Such systems not only amplify historical biases but are often opaque, even to experts. Furthermore, when discriminatory decisions occur, there’s often no feedback or recourse mechanism.

The logic behind algorithms disproportionately affects marginalized communities and minorities while further enriching the privileged, perpetuating systemic inequalities. While some cases, like the Kinect or Zoom failures, cause minor harm, others, such as recruitment or diagnostic algorithms, can lead to more severe consequences.

Towards Trustworthy Artificial Intelligence

Algorithms undoubtedly offer significant productivity gains, including in HR. However, if not guided by appropriate human oversight, they can completely undermine DEIB (Diversity, Equity, Inclusion, and Belonging) goals. Building trust in AI starts with recognizing and addressing algorithmic biases. Below are the foundational principles of responsible AI use, most of which align with the EU AI Act regulations that came into effect on August 1, 2024:

1. Transparency and Explainability: Clearly define where and how AI is applied, what data it relies on, and how that data is collected and filtered.
2. Accountability and Governance: Establish clear accountability and governance protocols for addressing errors.
3. Fairness: Regularly audit AI systems to ensure they provide equitable outcomes for different demographic groups.
4. Diversity and Inclusion: Employ diverse teams to better detect and address potential biases.
5. User Awareness: Raise awareness about potential biases in AI systems.
6. Ethical Guidelines: Implement ethical codes to define and manage acceptable risks.
7. Continuous Improvement: Incorporate feedback to refine systems.
8. Regulatory Compliance: Monitor changes to AI and GDPR regulations and ensure compliance.
9. Independent Expert Reviews: Regularly evaluate algorithmic risks and impacts with the help of external, independent experts.

DEIB strategies should be based on thorough data analysis, even when planning to introduce an AI solution. Data analysis can reveal hidden biases. The devil is in the details, and in the age of AI, this is particularly critical.

Comprehensive data analysis goes beyond simple surveys or organization-wide pay gap studies. Discrimination often doesn’t stem directly from explicit gender or racial bias but from hidden disparities that become discriminatory through deeper proxies. A typical example is the gender pay gap, which may look acceptable on the surface but reveals a more nuanced picture upon deeper analysis.

Artificial intelligence has immense potential in HR, but if biases are not consciously addressed, it risks completely undermining DEIB objectives. The solution lies in a balanced combination of technology and human oversight. Only then can AI become a true tool for progress.