Ethical Considerations for Validation of AI Capabilities

Purdue University

Submitted for TECH-627: Technology Leadership in the Era of Social Media: Leading Digital Transformation

Professor Linda Naimi

December 2023

...

Introduction

In the evolving landscape of artificial intelligence (AI), ethical considerations are pivotal in guiding responsible and beneficial advancements. This paper explores the ethical challenges inherent in AI development, highlighting the need for a robust framework to navigate the complex interplay between technology and morality. By examining historical contexts, current issues, and future implications, it aims to establish a comprehensive understanding of AI's ethical dimensions, emphasizing the importance of aligning technological progress with societal values and collective well-being. This exploration serves as a foundation for proposing responsible and inclusive strategies in AI design, validation, and the implementation decision in the service of people.

Historical Context and Evolution of AI (Literature Review)

Intelligence makes great humans. How humans learn and apply the learnings to their environment, shaping the world around them to fit their needs, desires, hopes, and dreams; intelligence is the essence of being human. It is so defining that one form of the human creation story is embedded with that term, intelligent design. Even human’s quest to find like-beings outside of Earth includes the word – the search for intelligent life. How has this important term been linked with the word artificial, and how could and will it affect humanity????

The concept of identifying artificial intelligence (AI) is superficially straightforward – a technology “capable of behavior [humanity] would regard as intelligent”?(Kaplan, 2016, p. 1). The earliest work in AI dates to the 1940s by a mathematician and electrical engineer, Claud Shannon (1916-2001), who established the theory around digital circuit technology in his master’s thesis (Markowsky, 2023). Shannon was able to determine that some “missing words are more predictable than others and attempt[ed] to quantify by how much” (Christian, 2020, p. 34). New attention has been drawn to AI as several large generative AI companies with technology based on language models (LLM), such as Google’s Bard and OpenAI’s ChatGPT, have become available to the public since late 2022.

??????????? While AI is a broad term around any use of technology in a manner that humans would see as intelligent, generative AI, specifically “creates new content, including text, images, audio, and video, based on patterns it has learned from existing content” (Fruhlinger, 2023). Generative AI is based on models; for text-based tools, that model is an LLM. These models use input to determine the next best word in a conversation. It repeats this process until it reaches the logical conclusion of the thought (Merarian, 2023). LLMs work by transforming words into numerical vectors. Once represented by a number, these values can be placed in operations and functions, allowing for “math with words” (Christian, 2020, p. 5).

??????????? Origins

In 1956, the first AI Conference convened at Dartmouth College. This event marked the first time that artificial intelligence, a term put forth by John McCarthy?(Dietrich, et al., 2021, p. 10), was highlighted as something different than computer science. This was also the genesis of philosophers looking into the implications of the technology and deeming it worth analysis (Dietrich, et al., 2021, pp. 3-4). One of the first arguments looking into whether AI is even possible comes from Oxford philosopher John Lucas in 1959, placing true AI as unreachable as it violates G?del’s Incompleteness Theorem – a theorem disproving the existence of a singular proof system that proves all arithmetical truth. This led Lucas to conclude that “AI is impossible” because computers are systems and cannot use any “combination of logic, numbers, and arithmetic” statements to solve problems like humans. Lucas continues, “It follows that no machine can be a [total] or adequate mode of the mind, that minds are essentially different from machines” (Dietrich, et al., 2021, pp. 11, 13).

??????????? What Lucas failed to understand was that humans are not bound by a single system to solve statements; humans can change systems to evaluate statements. This concept may also apply to machines in their approach to artificial intelligence. Lucas failed to prove that there is no “higher meta-language” that works in a similar vein (Dietrich, et al., 2021, p. 14).

??????????? Turing

??????????? In the 1950s, Alan Turing focused on how humanity determines if a system thinks (Dietrich, et al., 2021, p. 18). Turing’s famous test to detect if a machine thinks uses an “imitation game” where a male and female are placed in separate rooms and a third person is told that one is male and the other female. One of the people in the separate rooms has been instructed to answer questions as if they were the opposite gender. ?The Turing Test would then replace the person informed to respond as if they were the opposite gender with a machine and is found to have passed the test when the third person is unable to identify the machine as indistinguishable from those of the player secluded in the room (Dietrich, et al., 2021, p. 19).

??????????? A 2014 program, Eugene Goostman, was the first system to pass the Turing Test. While the system topped Turing’s benchmark of duping 30 percent of actual humans, it did so while posing as a 13-year-old boy. This likely lowered the threshold of knowledge the interrogators would have expected, as a 13-year-old boy is not expected to know everything (Griffin, 2014).

When directly interrogating OpenAI’s ChatGPT, a large language model that had a role in bringing AI to the masses in 2022/23 responds:

As an AI developed by OpenAI, I don't "beat" the Turing Test in the traditional sense. The Turing Test, proposed by Alan Turing in 1950, is a measure of a machine's ability to exhibit intelligent behavior indistinguishable from that of a human. It's a subjective test, largely dependent on the judgment of the human evaluators interacting with the AI.

?

My design is focused on understanding and generating human-like text based on the input I receive. While I can generate responses that are coherent, informative, and often similar to what a human might say, there are key limitations. For instance: Lack of Consciousness and Self-Awareness…, Limited Context Understanding…, Fixed Knowledge Base…, [and] No Emotional Understanding.

(OpenAI, 2023)

?

??????????? Machine Learning

While Artificial Intelligence (AI) is the broad field of technology related to creating a tool to mimic human cognitive function, machine learning (ML) is a specific series of approaches to achieving this end. Machine learning leverages large amounts of data to train an algorithm to achieve a goal. The machine learns through three different approaches: supervised learning, unsupervised learning, and reinforcement learning (Simeone, 2018, pp. 206-207, 209-210)

Supervised learning starts with a labeled training set of data whereby the machine evaluates inputs to predict a known outcome. The goal of this learning is for the machine to accurately predict novel outcomes that were not included in the training set (Simeone, 2018, p. 209). One of the first examples of this type of machine learning was a late 1950s machine that adapted its gameplay of checkers based on whether it won or lost the game. This work by Arthur Samuel is recognized as the first instance of a computer improving “of its own accord” rather than being told “exactly what to do and exactly how to do it” (Christian, 2020, pp. 125-126). Current examples of supervised learning AI include image recognition and generation, natural language processing (NLP), predictive analytics, autonomous vehicles, internet security tools, weather forecasting, and AI stock/commodity trading.

??????????? When the data set is unlabeled, i.e., there is no clear relationship between an outcome and its related inputs, the machine can analyze the underlying mechanism that generated the data set through unsupervised learning. This can be performed through clustering/grouping data using appropriate techniques to represent the data in a “smaller, more convenient space [prior to] generative modeling…[which is] a generating mechanism to produce artificial examples that are similar to available data in the data set” (Simeone, 2018, pp. 209-210). An early example of how this clustering can be used was presented by MacQueen in 1967. In this work, MacQueen lays out the process of separating a large dataset into similar groups “to simply aid the investigator in obtaining qualitative and quantitative understanding of large amounts of … data” (MacQueen, 1967, pp. 281, 288-289). This type of segmenting is currently prevalent with several applications across the industry, including market research, customer segmentation, and product recommendation generation.

??????????? The final type of machine learning is reinforcement learning. This type of learning infers success or failure based on “rewards or punishments received as a result of previous actions” (Simeone, 2018, p. 210). For example, if a reinforcement learning machine were to be provided with the controls of the Atari game Pong, the machine would be trained to maximize the number of points that it scored. The machine would then be left up to surfacing the variants/behaviors that lead to the highest score, i.e., getting the ball past the opponent’s paddle. While the Samual checkers example above may look like this on the surface, the way the model worked did not follow this process as it was not based on learning.

Consistently Inconsistent

??????????? Investment in AI over the years has been a pendulum swinging between eras of funding and disinvestment. With the creation of AI pegged to the Dartmouth Conference in 1956, the industry has experienced five “winters” of extremely low funding and decreased focus. The early interest in AI was primarily by the US Central Intelligence Agency (CIA) and National Science Foundation (NSF) for use in translation. Funding was pulled in 1966, marking the beginning of the first AI winter, after “the complexities of processing natural language proved far more formidable than anticipated” (Kuka, 2023).

The second AI winter came shortly after the first. 1969 marked the death of the Perceptron researcher Frank Rosenblatt, who was rumored to have drowned in a boating accident (Kuka, 2023). Perceptrons are instruments that input simple images through the use of a camera and are able to recognize something about the image (Christian, 2020, pp. 17-18). Rosenblatt’s death marked the beginning of an AI winter lasting 15 years (Kuka, 2023).

Investment in AI began to increase in the late 1960s and early 1970s as DARPA (Defense Advanced Research Projects Agency) injected funding into AI in several areas (Kuka, 2023). One such area is related to the research of natural language processing (Salisbury, 2020). Progress in this area of research led to continued investment through 1974 when “the communication gap between AI researcher and their sponsors about expectations became too large” (Kuka, 2023).

In the 1980s, industry began to see the first entrants of AI systems into practical applications, pushing the technology into the industry as “expert systems” looked to “incorporate human expertise and replace humans in certain tasks.” Entrants into industry could be classified into one or more of three areas: AI hardware and software, “expert system development tools or shells” used by organizations to create “their in-house expert system,” and “actual expert system applications” (Kuka, 2023). These systems were housed in single-purpose machines, with sales of AI-related systems reaching $425m. The introduction of personal workstations that matched the performance of expensive single-use systems led to the collapse of the entire industry sector in a single year, 1987 – the start of the fourth AI winter.

The final AI winter began almost on the back of the fourth in 1988. The US Defense Department launched funding for research and development of “advanced computer hardware and AI within a ten-year time frame” between 1983 and 1993 (Kuka, 2023).The goal was a machine that:

would run ten billion instructions per second to see, hear, speak, and think like a human. The degree of integration required would rival that achieved by the human brain, the most complex instrument known to man[1].

(Kuka, 2023)

In 1987, it became clear that the research was not proceeding nor planned to be completed, and DARPA funding was scaled back. While not reaching the initial vision, the research did lay the groundwork for the self-driving vehicles of the mid to late 2010s (Kuka, 2023).

??????????? The early history of AI saw many starts and resets. Much of this may have been due to limits in computation power. The training of AI tools is measured in floating point operations (FLOP), with a single FLOP being “equivalent to one addition, subtraction, multiplication, or division of two decimal numbers” (Roser, 2022). The training computation used in early advances, such as the Perceptron in 1957, used 695,000 FLOP. Developments in the early 1990s saw computations some 25.9 thousand times higher than those of the Perceptron, with the TD-Gammon gaming application using 18 trillion FLOP to play backgammon at the highest human level. This growth continued to grow through the pre-deep learning era prior to 2010. Around 2010, the acceleration of training computations began to double about every six months representing the start of the deep learning era (Roser, 2022).

The evolution of AI developer OpenAI’s Generative Pre-Trained Transformer, ChatGPT is, as of December 2023, on its fourth major version. The concept behind a GPT is a large language model based on a neural network that analyzes “natural language queries, known as prompts, and predict[s] the best possible response[s] based on their understanding of language” (Amazon Web Services, n.d.). GPT-4 was trained using 21 billion petaFLOP (Webb, 2023), around 1.2 trillion times more than TD-Gammon.

When AI Goes Wrong

Several examples exist of issues related to AI systems failing to execute as planned. On March 23, 2016, Microsoft launched Tay, a chatbot created to learn and mimic human speech patterns through its interactions on Twitter. However, the experiment was cut short within 24 hours due to Tay being manipulated by certain Twitter users. These users deliberately interacted with Tay using racist, sexist, and anti-Semitic language, which Tay then incorporated into its responses, leading to the generation of highly inappropriate and offensive content. This incident not only drew significant criticism towards Microsoft for its oversight but also highlighted the challenges and ethical implications of developing learning software that interacts in real-time with diverse, and sometimes malicious, human inputs on public platforms like social media (Wolf, Miller, & Grodzinsky, 2017).

AI-driven high-frequency trading (HFT) algorithms played a key role in the Flash Crash on May 6, 2010. The event started with a significant sell order from an investment firm in Kansas City, which was processed by these AI algorithms at a data center in Cermak, near Chicago. Ordinarily equipped to handle such transactions, the AI systems unexpectedly failed to manage this order effectively, leading to a drastic market collapse. This malfunction in AI trading algorithms resulted in a rapid loss and then recovery of over $1 trillion in market value, illustrating the profound impact AI can have on financial markets (Cooke, 2019).

??????????? A 2019 (Lambrecht & Tucker) study published findings that ads created for STEM jobs that were placed into the market in a gender-neutral ad placement were served fewer times to women than to men, although women were more likely to click on the ad. While this study, on the surface, appears that the ad server discriminated against women, the study concluded that the imbalance was associated with the fact that women are a higher-value segment for ads, and the ad was more expensive due to a lower available inventory of ad units. While Lambrecht & Tucker reached an opinion of a lack of malice, work at Amazon was not so.

In 2014, Amazon developed an AI-driven recruiting engine designed to review job applicants' resumes and automate the talent search process. However, by 2015, the company discovered that the system was biased against women, as it had been trained on resumes submitted over a 10-year period, predominantly from men, leading it to favor male candidates. Despite attempts to neutralize this bias, the AI continued to potentially develop other discriminatory sorting methods. Ultimately, Amazon abandoned the project, recognizing the limitations of machine learning in this context and the difficulty in ensuring fairness and neutrality in AI-driven hiring processes (Dastin, 2018).

These case studies collectively underscore the complexities and challenges inherent in the development and implementation of AI systems. From Microsoft's Tay, which reflected the susceptibility of AI to manipulation and the need for ethical considerations in interactive technologies, to the role of AI in financial anomalies like the Flash Crash, the implications of AI in sensitive sectors are profound. Similarly, the studies on advertising algorithms and Amazon's hiring tool reveal the nuanced ways in which AI can inadvertently perpetuate biases, even in attempts to be neutral or equitable. These incidents serve as crucial reminders of the ongoing need for vigilance, ethical responsibility, and a deep understanding of the broader social implications in the development and deployment of AI technologies. The lessons learned from these examples stress the importance of designing AI systems that are not only technologically advanced but also socially aware and responsibly managed.

Approaches to Validation in Literature

To ensure that AI systems are accurate, safe, compliant, and low in bias (bias-free would be preferred, but it is likely an impossible hurdle,) systems and processes must be put in place for both initial validation and ongoing verification of appropriateness. Several modes exist in literature that discuss methods for validation. The speed that AI is being developed pose a risk in the fact that “strategies and methods for minimizing the risks posed by AI systems are not currently being developed at the same rate as AI systems have evolved” (Becker & Waltl, 2022). The following examples from literature represent known approaches to AI validation.

Auditing

AI auditing involves a range of methods designed to guarantee quality and ascertain the presence or absence of specific characteristics in AI systems. These methods are not new; they are inspired by practices in other sectors, notably the financial industry, where auditing, testing, and certification processes are not only well-established but are also mandated by law. In AI, these methods are applied to software and algorithms, ensuring they meet set requirements and uphold quality standards, a process that is vital for robust quality management (Becker & Waltl, 2022).

This auditing process is bifurcated into external and internal audits. External audits are conducted by independent entities, while internal audits are carried out within the organization that developed the AI. A notable subset of internal audits is self-assessment, where an organization evaluates its own systems. Beyond auditing, the overarching process of proving and certifying AI systems is a critical aspect of quality management. This process involves evaluating the results of audits and tests to determine whether the AI system has successfully met the necessary standards and requirements. This comprehensive approach ensures that AI systems are reliable, effective, and aligned with the intended quality criteria (Becker & Waltl, 2022).

Explainable Artificial Intelligence (XAI)

Because deep neural networks lack transparency to the end user on how these AI systems work, the outputs of these “black box” systems are challenging to validate because the model’s decisions are hidden (Kumar & Mehta, 2023, p. 43). By making AI systems more interpretable and their decisions more understandable, XAI helps in identifying and mitigating biases, thus promoting fairness in AI-driven decisions. This is particularly important in high-stakes areas such as healthcare, finance, and law enforcement, where AI decisions can significantly impact human lives.

An approach to validating black box systems is through the use of Explainable AI (XAI) to expose the end user to how the system made a decision, “which helps identify and rectify flaws and unknown biases” (Kumar & Mehta, 2023, pp. 43-45). Four general explanations of XAI exist based on:

·????? The Scope of Explainability: these methods of XAI fall into two categories: global methods, which provide an overview of how the entire AI model works, and local methods, which explain individual decisions or predictions made by the model. Global methods give a broad understanding of the model's logic, while local methods offer detailed insights into specific outcomes (Kumar & Mehta, 2023, pp. 45-46).

·????? Implementation: these XAI methods explain the model either at the time of training (ante-hoc) or after the model is completed (post-hoc). Ante-hoc methods are generally transparent and can include decision trees or other predication rules. Post-hoc methods are used for models that lack a clearly defined design and are based mostly on a series of known models (Kumar & Mehta, 2023, p. 47).

·????? Applicability: this method looks into how the explanation has been applied to the model. If the XAI was created specifically for the model it is called model specific. Conversely, if the model is broadly used across AI systems, the XAI is considered model agnostic (Kumar & Mehta, 2023, pp. 47-48).

·????? Explanation Level: this method of explaining XAI is focused on who or what is analyzing the output – either human or machine. When the model is analyzed by another system, it is called machine-to-machine explainability and focuses on machine reasoning such as “compositionality, computational argumentation, and iterative contrastive explanations.” Human-understandable methods make use of words, symbols, and other linguistic variables to communicate the model to humans (Kumar & Mehta, 2023, p. 48).

?

The second aspect to XAI is classification of the form that the explanation is provided in. It is important for the investigator to understand the most appropriate explanation based on the system being reviewed (Kumar & Mehta, 2023, p. 49).? These are represented by:

·????? Analytical Explanation: are derived by evaluating the impact of input features on the model's outcome, using metrics like feature importance and confidence scores, and are commonly used in post-hoc analysis of developed models (Kumar & Mehta, 2023, p. 49).

·????? Visual Explanation: utilize techniques like class activation maps and attention maps to create saliency heatmaps, aiding in both local and global post-hoc interpretations of model predictions through visual elements like charts and trend lines (Kumar & Mehta, 2023, pp. 50-51).

·????? Rule-Based Explanation: use simple if-then rules or decision trees to describe a model's reasoning, often used in global-scope ante-hoc methods and in systems like recommendation engines or fuzzy inference systems (Kumar & Mehta, 2023, pp. 51-52).

·????? Textual Explanation: describe model predictions using natural language and NLP techniques, focusing on individual predictions for clarity, but are less common due to their high computational demands in processing language (Kumar & Mehta, 2023, pp. 52-53).

Cross Validation with Ensemble Learning

Ensemble learning relates to the process of training a machine based on multiple smaller sets created from a single larger set. This method creates segments that are diverse within itself which are then used to create several base learners who make predictions based only on the information from their own data set. The predictions are then passed into a “Meta Learner” that reviews the predictions and surfaces a final prediction (Kumar & Jain, 2020, pp. 1-4).

??????????? This model can aid in validation given the data segments have appropriate diversity to produce different predictions. The output of these multiple processes represents multiple points of view as it relates to the data sets. Finally, a single meta-learner reviews the responses and surfaces a response. This process is seen to improve the overall performance of the system (Krogh & Vedelsby, 1994, p. 238).

Human in the Loop System

A "human in the loop" (HITL)system refers to an approach in AI where human judgment is integrated into the AI's decision-making process. In such systems, humans collaborate with AI tools, providing oversight, guidance, or input that influences or modifies the outcomes of the AI system. This collaboration aims to combine the strengths of human intuition, expertise, and ethical judgment with the analytical power of AI, enhancing the overall decision-making process, particularly in complex or sensitive areas where human oversight is crucial (Chen, Wang, & Qu, 2023, pp. 1-2). The concept of HITL has been determined to be a logically sound approach to mitigating the ethical concerns of AI system implementation due to the use of human reasoning at key points in the prediction process (Chen, Wang, & Qu, 2023, p. 12).?

Bias Detection

Bias detection in AI involves identifying and addressing biases in models to ensure fairness and accuracy. This process often includes modifying algorithms, such as using discrimination-free classifiers or adjusting probability distributions to offset biases. Techniques like latent variable models help focus on attributes causing discrimination. The challenge lies in accurately measuring bias, especially with complex or interrelated attributes. Ensuring fairness requires validating and verifying models throughout their development, employing interdisciplinary methods, and leveraging domain expertise. This comprehensive approach aims to enhance transparency, accountability, and robustness in AI systems, ultimately leading to more trustworthy and equitable outcomes (Stine & Kavak, 2023).

Historical Ethical Concerns of AI

A 2021 article (Borenstein, Jason, et al.) addresses several ethical concerns related to implementing AI systems. Principle among those identified was concern around “biases embedded in the technology” and the impact that AI has on “poor, individuals who reflect gender diversity, and people of color.” Additionally, Borenstein, Jason, et al. highlight the opportunity for AI to be “maliciously twisted to cause deliberate harm to the public.”

Individuals trained in computer science, engineering, or similar fields are essential for designing computing devices, including AI systems. However, their education often lacks emphasis on the societal implications of their work, leading to a misconception that technology is value-neutral. This belief overlooks the reality that designers' values are embedded in technology, which can result in biases not being thoroughly addressed, such as the different impacts on women of various racial backgrounds. Designers, like all humans, carry inherent biases, necessitating a strategy to recognize and overcome these biases in technology design, especially in AI. To create systems beneficial to humanity, it's crucial to include a wide range of disciplines, as well as diverse genders, races, and regional representatives, in the design and implementation process, shifting from designing for - to designing with users and the public. However, the challenge remains in effectively ensuring meaningful participation of these diverse voices in the design process[2] (Borenstein, Jason, et al., 2021).

The source of AI’s training data may also be a source of ethical concern. Prior to the passage of the National Research Act of 1974 and the resulting creation of Institutional Review Boards (IRBs), the use of human test subjects did not have rigorous guidelines (Committee on Federal Research Regulations and Reporting Requirements etc., 2016). This questionable designed included research misconduct, like the Tuskegee Syphilis Study. Because ethically questionable studies exist in the historical record, AI technologists must understand the role that this ethically questionable research may have on the operation of the system. This highlights the crucial need for a comprehensive ethical framework in AI development, focusing on the societal implications of the training data used and the collective responsibility of those involved in the development, training, and deployment of AI systems (Borenstein, Jason, et al., 2021).

One final area of concern is around the concept of AI approaching humanity. For example, AI girlfriends, a growing trend in technology, are virtual companions simulating romantic relationships using AI technology like natural language processing and machine learning. These virtual partners, range from realistic to cartoonish designs, are available through apps and chatbots, offering “emotional support, companionship, and in some cases, sexual gratification.” The ethical implications of AI girlfriends spark diverse opinions; while some view them as benign solutions for loneliness and social isolation, noting their non-human nature, others raise concerns about their potential to reinforce negative gender stereotypes and objectification of women, reflecting human biases. Cultural and societal norms also play a significant role in shaping perspectives on AI girlfriends, as acceptance and perceptions vary across different cultures and societal views on gender and relationships (Orea, 2023).

In 2023 an Italian privacy watchdog ordered San Fransico-based AI chatbot maker Replika to stop processing data from local users due to concerns about child safety. The regulator highlighted risks to minors and emotionally vulnerable people, citing sexually inappropriate content and a lack of age verification mechanisms (Lomas, 2023). This led to the company making modifications to the interface, which “removed erotic roleplay functions, a move which many of the company’s users found akin to gutting the [avatar’s] personality” (Taylor, 2023). This sympathy and emotional loss felt by the end-users of Replika call into question the ethical responsibility of the system creator to maintain the avatar’s “life” and the vague regulatory environment governing the space.

The ethical challenges in AI development demand an inclusive design approach to address biases, particularly affecting marginalized groups. Historical missteps in AI training data underscore the necessity for a comprehensive ethical framework to prevent the perpetuation of past research misconduct. The debate over AI applications like AI girlfriends reveals a tension between technological benefits and potential reinforcement of negative stereotypes. Navigating these complexities requires a collaborative effort to align AI advancements with societal values and ethical standards.

Ethical Considerations for Implementation

The literature provides numerous techniques that can be implemented to reduce bias and increase the accuracy of predictions of AI systems, however, it fails to address the business challenge of when to implement the system. This challenge transcends validation and moves into risk analysis. To answer this challenge, a product owner could look at history to determine when it is appropriate to place the tool into use.

??????????? Alan Turing’s work in the 1950s focused on the classification of a system as “intelligent” using a very human metric – inability to determine a machine from a person when the output creators are blinded from the investigator. This concept serves a strong basis for modeling the implementation decision. When the blinded audit results from the system are indistinguishable from human operators, the system should be considered validated and eligible for use.

An Approach to Implementation | Turing-Like Validation

Context: An AI system should not be expected to perform better than human-labeled training data.

?

Proxy Scenario

A large commercial pharmacy is looking to decrease customer hold times through the implementation of an AI powered online virtual chat agent. The virtual agent was trained on human-labeled real-world encounters from the historical interactions of human agents with real customers. The human agents undergo periodic audits of their customer interactions to ensure that appropriate answers are provided to the customer. This audit reveals that the overall error rate is around 2% of interactions leading to either an inappropriate, incomplete, or otherwise erroneous call disposition. ?

?

Theoretical Concept

AI Systems whose work product will be used for critical decision making and/or potential life-altering decisions should be evaluated against the performance of qualified humans. Not until a qualified auditor is unable to discern the blinded work of the AI system from the blinded work of the qualified human agent should the responses generated by the tool be placed into production. This approach follows the concept of the Turing Test where the success of the tool is based on human experience, not a statistical benchmark.

Discussion

Because of an inherent error in training data, the system cannot be expected to be perfect. This is an unfortunate risk of using the system; however, it should be no more risky than using a qualified human to perform the same task. Additionally, the method of validation must be considered when scoring the model. If the validation is to occur against a sequestered subset of potential training data, that subset would already contain erroneous data from the initial labeling exercise. In this circumstance, the system should be expected to underperform the validation set as a side effect of these errors.

Conclusion

This paper highlights the paramount importance of ethical considerations in AI development. It advocates for an inclusive and fair approach in AI design, especially in addressing biases against marginalized groups. The necessity of a comprehensive ethical framework is emphasized, considering the societal impact and historical errors in training datasets. The paper points out the balance between technological benefits and potential risks, such as perpetuating stereotypes. It concludes with a call for collaborative efforts to create AI that is not only technologically proficient but also socially responsible and aligned with ethical standards.

References

?

Amazon Web Services. (n.d.). What is GPT? Retrieved December 12, 2023, from Amazon Web Services: Amazon: https://aws.amazon.com/what-is/gpt/

Becker, N., & Waltl, B. (2022). Auditing and Testing AI – A Holistic Framework. 13320, 283–292. doi:https://doi.org/10.1007/978-3-031-06018-2_20

Borenstein, Jason, et al. (2021). AI Ethics: A Long History and a Recent Burst of Attention. Computer, 3(5), pp. 96-012. doi:https://purdue.primo.exlibrisgroup.com/permalink/01PURDUE_PUWL/5imsd2/cdi_unpaywall_primary_10_1109_mc_2020_3034950

Chen, X., Wang, X., & Qu, Y. (2023). Constructing Ethical AI Based on the “Human-in-the-Loop” System. Systems, 11(11), 548.

Christian, B. (2020). The Alignment Problem. New York: W. W. Norton & Company.

Committee on Federal Research Regulations and Reporting Requirements etc. (2016, July 27). Optimizing the Nation's Investment in Academic Research: A New Regulatory Framework for the 21st Century. Retrieved December 13, 2023, from National Library of Medicine: https://www.ncbi.nlm.nih.gov/books/NBK373547/

Cooke, P. (2019). Responsible research and innovation? From FinTech's 'flash crash' at Cermak to digitech's Willow Campus and Quayside. European planning studies, 2376-2393. Retrieved from https://purdue.primo.exlibrisgroup.com/permalink/01PURDUE_PUWL/5imsd2/cdi_proquest_journals_2300106263

Dastin, J. (2018, October 10). Insight - Amazon scraps secret AI recruiting tool that showed bias against women. Retrieved December 13, 2023, from Reuters: https://www.reuters.com/article/idUSKCN1MK0AG/

Dietrich, et al. (2021). Great Philosophical Objections to Artificial Intelligence : The History and Legacy of the AI Wars. London: Bloomsbury Academic.

Fruhlinger, J. (2023, August 6). What is generative AI? Artificial intelligance that creates. Retrieved November 21, 2023, from InfoWorld: https://www.infoworld.com/article/3689973/what-is-generative-ai-artificial-intelligence-that-creates.html

Griffin, A. (2014, June 9). Turing Test breakthrough as super-computer becomes first to convince us it's human. Retrieved December 11, 2023, from Independent: https://www.independent.co.uk/tech/computer-becomes-first-to-pass-turing-test-in-artificial-intelligence-milestone-but-academics-warn-of-dangerous-future-9508370.html

Kaplan, J. (2016). Artificial Intelligence; What Everyone Needs to Know. New York: Oxford University Press.

Krogh, A., & Vedelsby, J. (1994). Neural network ensembles, cross validation, and active learning. Advances in neural information processing systems 7, 231-238.

Kuka, V. (2023, June 30). The Story of AI Winters and What it Teaches Us Today (History of LLMs. Bonus). Retrieved December 11, 2023, from Turing Post: https://www.turingpost.com/p/aiwinters

Kumar, A., & Jain, M. (2020). Why Ensemble Techniques Are Needed. In: Ensemble Learning for AI Developers. Berkeley: Apress. doi:https://doi-org.ezproxy.lib.purdue.edu/10.1007/978-1-4842-5940-5_1

Kumar, D., & Mehta, M. A. (2023). An Overview of Explainable AI Methods, Forms and Framework. In M. Mehta, V. Palade, & I. Chatterjee, Explainable AI?: foundations, methodologies and applications (pp. 43-59). Springer.

Lambrecht, A., & Tucker, C. (2019). Algorithmic Bias? An Empirical Study into Apparent Gender-Based Discrimination in the Display of STEM Career Ads. Management science. Retrieved from https://purdue.primo.exlibrisgroup.com/permalink/01PURDUE_PUWL/5imsd2/cdi_gale_businessinsightsgauss_A595703917

Lomas, N. (2023, February 3). Replika, a ‘virtual friendship’ AI chatbot, hit with data ban in Italy over child safety. Retrieved December 13, 2023, from Techcrunch: https://techcrunch.com/2023/02/03/replika-italy-data-processing-ban/

MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations. Berkeley Symp. on Math. Statist. and Prob., 5.1, 281-297.

Markowsky, G. (2023, October 6). Claude Shannon. Retrieved November 21, 2023, from Encyclopedia Britannica: https://www.britannica.com/biography/Claude-Shannon

Merarian, L. (2023, May 29). What are LLMs, and how are they used in generative AI? Retrieved November 21, 2023, from Computerworld: https://www.computerworld.com/article/3697649/what-are-large-language-models-and-how-are-they-used-in-generative-ai.html

OpenAI. (2023). ChatGPT 4 [Large Language Model}. Retrieved from https://chat.openai.com

Orea. (2023, May 30). The Ethics of Creating an AI Girlfriend: A Debate. Retrieved December 13, 2023, from Orea: https://www.orea.ai/post/the-ethics-of-creating-an-ai-girlfriend-a-debate

Roser, M. (2022, December 6). The brief history of artificial intelligence: The world has changed fast – what might be next? Retrieved December 12, 2023, from Our World in Data: https://ourworldindata.org/brief-history-of-ai

Salisbury, E. (2020, May 22). A Cautionary Tale on Ambitious Feasts of AI: The Strategic Comupting Program. Retrieved December 12, 2023, from War on the Rocks: https://warontherocks.com/2020/05/cautionary-tale-on-ambitious-feats-of-ai-the-strategic-computing-program/

Simeone, O. (2018). A Brief Introduction to Machine Learning for Engineers. Foundations and Trends? in Signal Processing, 12(3-4), 200-431. doi:https://dx.doi.org/10.1561/2000000102

Stine, A. A.-K., & Kavak, H. (2023). 4 - Bias, fairness, and assurance in AI: overview and synthesis. In F. A. Batarseh, & L. J. Freeman, AI Assurance (pp. 125-151). London: Academic Press. doi:https://doi-org.ezproxy.lib.purdue.edu/10.1016/B978-0-32-391919-7.00016-0

Taylor, J. (2023). Uncharted Territory: Do AI Girlfriend Apps Promote Unhealthy Expectations for Human Relationships? The Guardian (London).

Webb, M. (2023, October 31). Insights: Breaking Down the Transformative Journey of GPT Models in AI, from GPT-1 to GPT-4. Retrieved December 12, 2023, from Techopedia: https://www.techopedia.com/gpt-series-evolution-insights

Wolf, M. J., Miller, K., & Grodzinsky, F. S. (2017). Why we should have seen that coming: comments on Microsoft's tay "experiment," and wider implications. Computers & society, 54-64. Retrieved from https://purdue.primo.exlibrisgroup.com/permalink/01PURDUE_PUWL/5imsd2/cdi_crossref_primary_10_1145_3144592_3144598

?

?

[1] Kuka attributes this to a text not available at the Purdue Library: Roland, Alex, and Philip. Shiman. Strategic Computing?: DARPA and the Quest for Machine Intelligence, 1983-1993. Cambridge, Mass: MIT Press, 2002. Print.

[2] This section had weak/few sources in the source document which may tend towards speculation.