What, How, and Why of Artificial Intelligence Risk Management

The Tenets of Artificial intelligence

All forms of?Artificial intelligence (AI)?can be risky, especially when newly implemented. This article addresses the risks and rewards of AI, and explains how?to?protect against risks. Not all risks are preventable, but they can be managed.?

AI has the potential to?transform society?for the better?and improve lives?in many ways. At the same time, however,?it?introduces?risks that can harm??not only business enterprises but the also the workers?and the?end users of business products. Security?risks?from AI?are different from those posed by traditional software or information-based systems.?This occurs because the?complexity??and relative novelty?of AI systems make it difficult to detect?risks?and respond?appropriately. We must all learn to anticipate?AI risk?and to effectively deal with it when it arises. We must learn to designing appropriate AI that aligns?with?the?goals and values?of specific businesses.?

Responsible use of AI means a razor focus on the human beings that run the business, on the community for whom the business is run, and?on the maintenance or?sustainability?ofthe?business for the foreseeable future. A human focus??necessitates a critical weighing of?potential positive and negative impacts of AI systems.??

First?and?foremost, are the individuals who may be affected. Next is the socialresponsibility?to one’s community?and?its prosperity.??All the while, there is an urgency to consider?and?not endanger?future generations.

Responsible?use of?AI??results?in?outcomes that are?equitable?for all. So, when?designing, developing, and deploying AI systems, please keep goals and values in mind.

A Risk Management Framework

An AI risk management framework (a “Framework”)?has?to minimize negative impacts of AI systems and maximize?its?positive impacts. This is done?by?awareness of risks,??screening for risks, meticulously documenting risks, and??effectively managing?them when they emerge.?A Framework?considers?the likelihood of?a risky?event occurring and?qualitatively and quantitatively measures its negative impact, direct and indirect.?

Building a risk management Framework takes time but is imperative because, if carried out properly, it protects against negative?consequences?for?the?firm, for?the?people working in the firm and for the community the firm serves. It may also protect future generations.??The human aspect is important because humans are fallible and make mistakes, are forgetful, are sometimes lazy, often don’t follow rules. To be trustworthy, AI systems have to take human frailty into account.?

Various other?factors complicate the?potential for?AI risks, including the use of third-party software, hardware, and data,??the advent of unexpected emergencies, the?availability of reliable metrics,?the?differences in risk assessment at different stages of the AI lifecycle, real-world deployment scenarios, and?difficulties in early understanding of signs of risk.

Third-party data or systems can facilitate the development and deployment of AI systems, but can also pose new risks that are difficult to measure. Emergent risks need to be identified and tracked. This indicates the need for?impact assessment approaches. Unfortunately, there is a??current?lack of consensus on robust and verifiable measurement methods for AI systems?that are trustworthy. Moreover, real-world scenarios often differ from the results obtained in a laboratory or controlled environment. The effects of an AI-enabled cyberattack?can,?for?instance, be very difficult to??anticipate and recognize. In?AI systems intended to augment or replace human activity,?it?is difficult to??compare a human response to an?AI?response because the tasks are?different.?

Risk Tolerance

Risk tolerance refers to the?severity of?risks??one is willing to bear?in order to achieve?one’sobjectives.?This?tolerance is influenced by legal and regulatory requirements, policies and norms established by AI system owners and policy makers, and can change over time as AI policies evolve. A Framework can prioritize risk, but it?cannot?dictate risk tolerance. Different organizations have different?degrees of??risk?tolerance?based on their?specificpriorities and resource considerations. A Framework is meant to complement?not replace?existing risk practices. In sectors where established guidelines for risk tolerance do not exist, organizations must define risk tolerance?that is reasonable for them?and use a Framework to manage?the?risks and document?their?risk management processes.

Eliminating all risks is unrealistic and can?actually?be counterproductive, so it’s important to recognize that not all AI risks are equal. A risk management culture can help organizations allocate resources?appropriately.?When applying a Framework, the level of risk and potential impact of each AI system should be assessed, and resources should be prioritized accordingly. Higher priority may be given to AI systems that interact directly with humans?or?involve sensitive data.?Lower priority?can?be given to AI systems that only interact with computational systems and?access?non-sensitive data. Regular risk assessment prioritization is important, even for non-human-facing AI systems that may have?potential?downstream impacts. Residual risk, or risk that remains after risk treatment,?must be?documented?in order to inform?end users about potential negative?consequences in?the?future.

AI risk management cannot be done in isolation and must be incorporated into a broader enterprise risk management strategy. A Framework should be used in conjunction with other relevant guidelines to manage AI risks?which include?privacy, cybersecurity, energy and environmental?impact?and?overall?security of AI systems. There should be clear accountability mechanisms, roles, and responsibilities for effective risk management, which requires commitment from senior?management. It is possible that this will??requirea cultural change within the organization or?within the?industry?as a whole.??The challenges for small to medium-sized organizations implementing a Framework may differ from those of large organizations?because of size differences in?resources and capabilities.

The following are characteristics of good AI risk management systems:

Trustworthiness

Trustworthiness is essential for AI systems?that employees accept?and?use?effectively.?Definition of?trustworthiness: it must be?valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair. All?harmful bias?must be?managed.?Achieving?trustworthiness is not just about fulfilling?each of?these characteristics, but?also about?balancing them, one against?the?other. Trade-offs may arise between different characteristics and the choice of which ones to prioritize depends on the context and?the values that are most important to the organization.

Involve multiple perspectives and parties in the AI lifecycle to ensure that decisions are informed and?applicable to a specific context.?Ultimately, trustworthiness depends on the decisions and actions of those involved and requires transparency and justification.?

Trustworthiness?characteristics of AI systems are interrelated.?Their?deployment?mustconsider the costs, benefits, impacts, and risks.

• Validation is the process of confirming that the AI system has met the requirements for its intended use.
• Reliability refers to the ability of the system to perform as required without failure over a given time interval.
• Accuracy is the closeness of results to the true values.
• Robustness refers to the ability of the system to maintain performance under different circumstances.

Validity, reliability, accuracy, and robustness should all be assessed through?routine?testing and monitoring?to minimize?potential negative impacts. In cases where the AI system cannot detect or correct errors, human intervention may?often?be necessary.

Transparency

Transparency refers to the extent to which information about an AI system and its outputs is accessible to those?who interact?with?it. Transparency enhances confidence in AI systems by promoting understanding?of its potential and of its limitations.??The more transparency, the less uproar about?negative impacts caused by?wrong?outputs. Transparency?helps in keeping responsible parties accountable??for severe consequences, such as those involving life and liberty. It?can be enhanced by?documenting and?maintaining training data,?knowing who is responsible for what?decisions, and?repeatedly?testing transparency tools with AI deployers.?Not everything can be shared –?proprietary information must, of course,?be?kept confidential.?

Risk to Human Life, Health, Property, or Environment

AI systems should not pose any risk to human life, health, property, or environment. Safe operation is?essential.?Different types of safety risks may need?to be?tailored?to?risk management approaches, with the highest priority given to those that pose serious injury or death risks. To prevent dangerous conditions, AI safety considerations should be incorporated into the system starting as early as possible in?the?planning and designphase. This incorporation may involve rigorous simulation and testing, real-time monitoring, and the ability to shut down, modify, or intervene in the system if it deviates from?what was?intended. AI safety risk management should align with existing sector or application-specific guidelines or standards and take cues from safety efforts in?other?fields.

Resilience

AI systems are considered resilient if they can withstand unexpected changes and?stillcontinue to function. They should be designed to degrade safely in case of failure. Security is?critically?important, and systems should have protection mechanisms in place to maintain confidentiality, integrity, and availability. Security and resilience are related but distinct, as security includes resilience and also involves avoiding and protecting against attacks. Resilience refers to the ability of the system to return to normal function after an unexpected event and covers robustness against unexpected or adversarial use of data.

Explainability

Explainability and interpretability are important characteristics of AI systems that ensure their?effective function?and?their?trustworthiness. Explainability refers to?how?the mechanisms behind AI?systems operate, while interpretability refers to the meaning of AI outputs?as they pertain to?their designed purposes. Together,?explainability and interpretability?help users and those in charge of AI systems understand the potential impact of the system.?A?lack of explainability and interpretability can?lead to misgivings and failure to use the systems appropriately.??Explainability and interpretability can be managed by providing?easy-to-understand?descriptions of AI functions and?also by clearly?communicating why decisions?about their use??were made. Transparency, explainability, and interpretability are distinct but interconnected?concepts, with transparency answering “what”, explainability answering “how”,?and interpretability answering “why”.

Privacy

Privacy refers to the norms and practices that help protect individual autonomy, identity, and dignity by limiting?surveillance and?intrusion, and?preventing?the disclosure of personal information. Privacy values should guide the design, development, and deployment of?all?AI systems. AI systems can pose new risks to privacy. Privacy-enhancing technologies and data minimizing methods such as de-identification and aggregation?are adesign?must.

Fairness

Fairness in AI refers to addressing issues of equality and avoiding?all forms of?bias and discrimination. The concept of fairness is?culture-specific?and?varies in different parts of the world.?Risk management in AI is enhanced by recognizing and considering?culturaldifferences.

There are three major categories of AI bias that organizations?must?consider and manage: systemic, computational,?statistical, and human-cognitive.?Biases?can be present in datasets,?in?organizational norms and practices, and, of course, in?the broader society?from which AI not only gleans its data but by which?AI systems?are used. Computational and statistical biases can occur in AI datasets and algorithms, often due to?sourcing datafrom non-representative samples. Human-cognitive biases relate to the way individuals or groups perceive AI system information and make decisions, or how they think about the purposes and functions of AI.?Simple examples are assumptions that all users are right handed, all have normal sight, normal hearing, and can nimbly operate keyboards.

Bias can exist in many forms, including unconsciously?ingrained in?early designs. Organizations need to consider?the?needs?of?all their employees and all the community users of their products.

Conclusion

AI must be based on principles of privacy, fairness and freedom from bias. It must be transparent, explainable, and easily interpretable. It must be resilient, trustworthy, reliable, valid, accurate, and robust. It must pose no risk to human life,??and minimal, preventable or manageable risk to health, property, or the earth’s environment.?While AI has great potential to transform society for the better, we must keep in mind the many issues discussed in this paper to ensure that?risk, while never possible to totally eradicate, is sufficiently mitigated to allow AI to bring significant improvement to people’s lives.