Navigating the Ethical Challenges of AI in Organizations

Introduction: Making AI Ethics Tangible

When thinking about the ethics of artificial intelligence, a term that often comes to mind is "squishy." It’s how a senior executive once described the topic, explaining why his organization wasn’t taking much action on it—it felt too vague, too hard to pin down. He saw himself as someone who deals with concrete facts, not something "fuzzy," "theoretical," or "subjective." To him, ethics was a messy stew, something to clean up and discard or ignore until it could be swept under the rug. But this perspective, shared by many in technical fields, misses the mark. Ethics isn’t squishy, and there’s a lot that can be done with it.

Ethics can be used to set clear objectives, develop strategies to meet those objectives, and implement practical steps to bring them to life. For those concerned about the ethical and reputational risks of AI—or aiming to lead in responsible AI—ethics can be woven into every aspect of an organization. The purpose here is to explore how to think about ethics, understand the specific ethical challenges of AI, and integrate these principles into operations in a meaningful way. This isn’t just about listing actions—it’s about learning how to approach the topic so the necessary steps become clear. By the end, the ethical landscape of AI will be visible, and the tools to navigate it will be in hand.

This perspective comes from over two decades of experience researching, teaching, and advising on ethics and AI ethics across various sectors, including large corporations, nonprofits, and startups in fields like healthcare, finance, and insurance. The goal has always been to empower others to make informed, thoughtful ethical decisions, drawing on real-world examples of challenges faced by organizations and how they’ve been guided toward safer outcomes.

Why AI Ethics Matters in a Business Context

Why focus on AI ethics, especially in a business setting? Why should board members, executives, product managers, engineers, data scientists, and students care—not just as individuals but in their professional roles? While ethics is undeniably important, what makes AI ethics a pressing issue that demands attention in both personal and professional capacities?

It’s useful to divide AI ethics into two categories: using AI to create positive social impact and using AI to avoid ethical pitfalls. The first approach focuses on leveraging AI for goals like improving education in underserved regions, finding new energy sources, or reducing poverty—worthy aims that deserve recognition. The second approach is about preventing ethical missteps while pursuing goals, whether those goals are noble or neutral. For instance, developing an AI to efficiently process thousands of résumés is a neutral task—it doesn’t earn moral accolades. But if that AI discriminates against certain groups, such as women or people of color, it becomes a serious issue. This second approach is about ensuring harmful outcomes don’t occur during the development, acquisition, or use of AI, regardless of the intended purpose.

Put another way, this second approach is about mitigating ethical risks. And with AI, there are many such risks to address. Several organizations have already faced the consequences of ignoring these risks. A self-driving car caused a fatal accident. A healthcare AI was investigated for prioritizing healthier white patients over sicker Black patients. A financial institution faced scrutiny for an AI that set lower credit limits for women than men, though it was later cleared after negative publicity. A major retailer abandoned an AI for résumé screening after two years because it couldn’t stop discriminating against women. Facial-recognition technologies have been banned in multiple cities and criticized widely. An AI used in criminal justice was found to systematically discriminate against Black individuals in risk assessments, influencing judicial decisions on sentencing and bail. The list of such incidents is long.

These cases highlight that ethical risks aren’t just morally wrong—they also bring reputational, regulatory, and legal challenges. What’s particularly concerning is that AI operates at scale, meaning a single failure doesn’t affect just one person—it impacts everyone interacting with the system. This could include everyone applying for a job, seeking a loan, crossing the path of a self-driving car, being captured by a facial-recognition camera, receiving a diagnosis from AI at a hospital, or being targeted by marketing algorithms. Given AI’s vast potential applications, its rapid scaling, and the wide range of ethical issues that can arise, critical questions emerge: How much time and money does it take to handle a regulatory investigation? How many millions are spent on fines for legal violations? How much does it cost to rebuild trust after a scandal?

This is why the topic is relevant to a broad audience. For board members and executives, the responsibility is to prevent ethical risks from damaging the organization’s reputation or triggering legal action. For product managers, engineers, and data scientists, the goal is to avoid creating systems that invade privacy, discriminate, or manipulate users. Even employees not directly involved in AI development likely don’t want to work for an organization that overlooks these issues.

Why AI Ethics Requires Special Attention

One might assume that existing corporate codes of conduct or regulations already address these concerns. Codes of conduct promote integrity and ethical behavior, and there are laws against discrimination and harm, such as those protecting pedestrians from self-driving cars. Does AI ethics really need a distinct focus?

Yes, it does. Codes of conduct guide individual behavior, helping employees understand what actions are acceptable. Most people know how to avoid unethical behavior, and training can reinforce this. But AI-related ethical risks don’t arise from intentional misconduct—they stem from a lack of foresight, inadequate monitoring of AI in real-world scenarios, and not knowing what to look for during development or procurement. While the risks themselves—discrimination, privacy violations, or harm—aren’t new, AI introduces new ways for these issues to manifest, requiring new strategies to prevent them.

The same reasoning applies to laws and regulations. Because AI creates new pathways for legal violations, new methods are needed to prevent well-meaning but risky actions. This is challenging. Some approaches to mitigating AI ethical risks can conflict with existing laws or be legally permissible but ethically problematic, leading to reputational damage. Organizations may face difficult decisions: deploy an ethically risky but legal AI, use an ethically sound but illegal AI, or refrain from deployment entirely.

Understanding AI: A Foundational Overview

To grasp AI ethics, it’s important to clarify what AI means in this context. In popular media, AI often refers to conscious, destructive robots with ulterior motives. In technical terms, this is known as artificial general intelligence (AGI), a highly adaptive form of AI that could set its own goals, multitask beyond human capabilities, and potentially pose existential risks. Some prominent figures have warned that AGI could threaten humanity’s future, but AGI doesn’t exist today, and many believe it’s a distant prospect, if it’s achievable at all.

What exists now is artificial narrow intelligence (ANI), which is far more limited. ANI is designed for specific tasks—like calculating insurance premiums—and is inflexible, pursuing only the goals set by its human creators. It lacks desires or autonomy. While powerful, it’s also limited in understanding. A phone calculator can perform complex math instantly but can’t grasp that two cookies are better than one, a concept a toddler might understand. That same toddler is also unlikely to mistake a soccer ball for a bald head, but an AI designed to track balls might make that error.

Most ANI today, particularly in business applications, relies on machine learning (ML), a type of software that learns from examples. Consider familiar software like word processors, email platforms, or video games—all built on computer code, often using algorithms. Algorithms are mathematical equations that take inputs, perform calculations, and produce outputs. For example, an insurance algorithm might use inputs like age, sex, and driving record to calculate a premium, with each factor weighted according to predefined rules.

Machine learning operates differently. Instead of programming explicit rules, ML learns from examples. Imagine training an ML algorithm to identify dogs in photos. You’d provide 1,000 dog images and instruct the algorithm to find the common pattern among them. It analyzes the images, identifies a "dog pattern," and uses that to evaluate new images. If you show it a new photo, it determines whether it matches the pattern. If it’s wrong, you can correct it, and it refines the pattern with more examples. This is why online platforms often ask users to identify images—like clicking on pictures with cars—to train their ML systems.

This learning-by-example method is transformative. Without ML, identifying a dog would require an impossibly detailed set of rules, such as “if it has two eyes, two ears, and they’re a certain distance apart, it’s a dog.” But such rules would fail to account for variations, like dogs missing an ear or similarities with other animals. ML simplifies this by letting the algorithm discover the pattern, often with greater accuracy.

So why is AI gaining so much attention now, when ML algorithms were developed in the 1950s? Two factors explain this: processing power and data availability. ML requires large amounts of digitized data to learn effectively, and the digital revolution, fueled by the internet, has provided an abundance of such data. Additionally, modern computers have the processing power to handle these vast datasets, unlike their predecessors in the 1950s. In short, older algorithms combined with massive digitized data and advanced computing power have driven today’s AI revolution.

The Primary Ethical Challenges of AI

Discussions about AI ethics often center on three key issues: biased AI, unexplainable algorithms, and privacy violations. These challenges are frequently highlighted because they’re directly tied to how machine learning functions. Let’s explore each in turn.

Privacy Machine learning relies on data, often personal data about individuals. To maximize accuracy, there’s a strong incentive to collect as much data as possible—about users, their friends, families, and beyond. This data is used not only for traditional analytics, like understanding customer profiles, but also for training AI systems. The more data an AI has, the better it performs, which drives companies to gather more, often leading to privacy violations. Moreover, AI can combine data from multiple sources to make accurate inferences about individuals that they may not want revealed, heightening privacy concerns.

Explainability Machine learning processes vast datasets to identify patterns and make predictions, such as the likelihood of missing a payment or clicking on an ad. However, these patterns are often so complex, involving factors beyond typical human consideration, that it’s difficult to explain why the AI made a specific decision. For instance, an AI analyzing dog photos does so at the pixel level, a process too intricate for humans to follow. This is often referred to as the "black-box" problem: an organization might not understand why its AI denied a loan, set a particular credit limit, or targeted a specific individual with an ad.

Bias Bias in AI is a widely recognized issue. AI can produce outputs that unfairly affect certain groups, even without any intent to discriminate. Consider an example where an AI was trained to screen résumés using a decade of hiring data, including which résumés were approved or rejected by human reviewers. The AI identified a pattern: the organization rarely hired women. As a result, it began rejecting résumés from women, picking up on indicators like participation in women’s sports. This bias could stem from discriminatory hiring practices, broader systemic issues in certain industries, or other factors, but the AI simply replicated the pattern. Despite attempts to correct it, the bias couldn’t be eliminated, leading to the project’s cancellation—a costly lesson in the importance of addressing ethical risks.

These three issues—privacy, explainability, and bias—are central to AI ethics because they stem from the mechanics of machine learning. However, they’re not the only concerns. Many ethical risks depend on the specific application of AI. For example, facial-recognition technology can undermine trust, cause anxiety, alter behavior, and erode autonomy through surveillance. Other issues include whether an AI’s design is manipulative, disrespectful, harsh, or imposes unreasonable burdens on users. An overemphasis on the three main challenges can lead to a narrow approach, where organizations believe they’re addressing AI ethics by focusing solely on bias or explainability. A more comprehensive perspective is needed to capture the full range of risks.

Structure vs. Content in AI Ethics Programs

A practical understanding of AI ethics requires distinguishing between Structure and Content. An AI ethics program needs a Structure—policies, processes, and defined responsibilities to identify and mitigate ethical risks. This is the "how": the mechanisms for detecting and addressing issues. The Content is the "what": the specific ethical risks the organization seeks to avoid, such as privacy violations, unexplainable outputs, or biased decisions.

To illustrate this distinction, consider an extreme scenario. Imagine an organization with an impeccable AI ethics Structure: clear roles for data collectors, engineers, and product managers; reliable channels for raising concerns; and a dedicated ethics committee. Now imagine that organization has deeply unethical goals, such as promoting racial bias. Their Structure ensures their AI favors certain groups, obscures decisions about marginalized communities, and builds surveillance tools targeting specific populations. The Structure is perfect, but the Content—the ethical values they prioritize—is deplorable. An effective AI ethics program requires both a robust Structure and ethical Content.

Focusing too heavily on the primary challenges can lead to an incomplete approach, overlooking other risks. A comprehensive program demands a thorough understanding of all ethical risks (Content) and a systematic method to identify and address them (Structure). Many organizations, if they engage with AI ethics at all, take a limited approach, concentrating on bias and using technical tools to detect and mitigate it. This is far too narrow.

What Lies Ahead

Discussions of AI ethics often begin by condemning biased, unexplainable, and privacy-violating AI, expressing frustration at the organizations responsible. That’s addressing the Content. The conversation then shifts to technical tools for mitigating risks, assuming the issues are well understood. This assumption is flawed. Even those deeply involved in AI ethics struggle to determine the right actions because they view ethics as vague and subjective. They’re trying to build a Structure without fully understanding the Content.

The argument here is that a comprehensive Structure cannot be created without a deep understanding of ethics—the Content. Once the Content is clear, the Structure becomes straightforward. Many ask in frustration, “How do we make AI ethics actionable? What do we do?!” The response is to take a step back: the confusion arises from not understanding AI ethics well enough. With a deeper understanding, the path forward becomes clear. Knowing the risks and their origins makes the necessary actions apparent.

This exploration aims to dispel that confusion. It will begin by transforming vague ethics into something concrete, addressing why ethics is often seen as subjective and why that’s a barrier to building an AI ethics program. It will then delve into the three main challenges—bias, explainability, and privacy—exploring their origins and significance. Next, it will guide the creation of an AI ethics statement that shapes actions, not just serves as public relations. Following that, it will outline the Structure of an effective, scalable AI ethics program, building on the earlier insights. Finally, it will focus on how product teams should approach Content to perform their roles effectively.

This is a substantial topic, but key takeaways will be provided to aid understanding. Grasping the reasoning behind them will reveal the AI ethics landscape clearly. The first step is addressing the misconception that ethics is subjective.

Key Points

• AI ethics can be divided into using AI for positive social impact and using AI to mitigate ethical risks.
• Ethical risks are not only morally wrong—they also pose reputational, regulatory, and legal threats, potentially costing millions and eroding trust.
• Corporate codes of conduct don’t address AI ethical risks, which arise from oversight, not intentional misconduct.
• Current laws and regulations don’t cover all ethical risks that can damage a reputation.
• Today’s AI ethical risks stem from artificial narrow intelligence, specifically machine learning.
• The primary challenges are privacy violations, explainability, and bias, but other risks emerge from specific AI applications like facial recognition.
• An AI ethics program needs both Structure (mechanisms to identify and mitigate risks) and Content (what the organization views as ethical risks). Understanding Content simplifies building Structure.

Conclusion: Two Insights

Picture attending a conference on AI ethics and joining a group at a cocktail reception already discussing the topic. The conversation is predictable. Buzzwords dominate: accountability, transparency, explainability, fairness, surveillance, governance, trustworthiness, responsibility, stakeholders, frameworks. Someone mentions "black box." Concerns about AI’s dangers follow—biased datasets, unexplainable algorithms, privacy invasions, self-driving cars causing harm. Skepticism sets in: “You can’t really define AI ethics,” or “You can’t plan for everything,” or “It’s just personal opinions on right and wrong.” Someone might ask, “How do you make ethical principles actionable?” or mention performance metrics. The group ends with shrugs, agreeing AI ethics is crucial but challenging.

But with a deeper understanding, the issues become clearer. They can be viewed from both a business and ethical perspective, revealing that AI ethics isn’t as difficult as it seems. When a colleague suggests, “We need to address AI ethics,” it’s clear what a meaningful statement looks like versus empty rhetoric. If someone claims, “AI ethics is a technical issue for the AI team,” that view’s limitations are apparent. If a company offers a “solution for responsible AI” or a “fix for bias,” it’s evident that software alone can’t solve everything.

There’s clarity on the role of people in defining fairness metrics, when explainability is critical, and the nuances of privacy beyond anonymity. It’s understood how to create effective AI ethics statements that employees take seriously, and that Structure—not just software—is needed to systematically address risks. The AI ethics landscape is now visible and navigable.

Here are two insights. First, there’s a broader lesson here. By focusing on certain sections and removing the AI context, the principles can apply to articulating and operationalizing ethical values in any organization—whether developing AI, working with emerging technologies, or selling everyday products. For those aiming to build an ethically sound organization, these ideas are directly applicable.

Second, this discussion isn’t just about AI ethics—it’s about the value of ethical inquiry and philosophical exploration. The process involves distinguishing between Structure and Content, separating different types of values, understanding harm versus wrongdoing, analyzing explanations, exploring privacy levels, balancing competing principles, and examining ethical questions in product development. Engaging with these concepts is a form of philosophical analysis. If these distinctions and analyses have illuminated the AI ethics landscape, they demonstrate the power of philosophy. Despite skepticism about its relevance, philosophy is vital for meaningful progress.