How do you address the potential bias and discrimination in your AI data and algorithms?

One of the main issues when discussing AI ethics, fairness, and bias is that most principles and guidelines, despite being correct and widely agreed upon, lack pragmatic solutions. It's relatively easy to identify sources of bias as most real-life datasets have bias at least to some degree. It's more difficult finding AI ethics and fairness solutions that help address bias. Some of the key toolkits used by data scientists: 1. AI Fairness 360 (by IBM) 2. Google's "What If" Tool 3. Microsoft's Fairlearn These tools allow analysts and data scientists to test datasets and algorithms across a wide range of metrics and use mitigation strategies to overcome bias and possible discriminations.

A few major issues. 1) Often forgotten when discussing Fairness, Ethics, and Bias are the significant effects that culture has on them. 2) Additionally, as ethics in general and AI ethics specifically aren’t generally required in every computer science curriculum, education is noticeably missing and desperately needed. 3) Like the nuanced changes of simple refined queries, false contextual perceptions or output of an AI are generally not refactored within systems and are not considered a priori which can be very dangerous, especially within critical systems as they use more and more expert algorithms. Training and education for all algorithm types, especially for the recursive nature of reinforcement learning is vital.

It is imperative to build AI solution that is "ethical, fair and unbiased" however, AI products are entirely dependent on "data". ChatGPT, for example, is trained on large dataset from public sources most of which are biased. Unless OpenAI hires an army of data engineers to remove bias in data, it is near impossible to eliminate it in AI. But it is possible to control for bias in production. My recommendations: 1) Disclose use of AI or 3rd-party AI, 2) Allow clients to Opt-In, 3) Communicate User Data being collected for use in AI, 4) Adhere to GDPR, 5) Audit data often, 6) Assess impact of AI on Humans, 7) Build Confidence Intervals with AI Output. My blog: https://www.dhirubhai.net/pulse/proposal-ai-transparency-kiran-sohi-mba/

Risk and impact analysis are part of AI best practices at every step of the AI development lifecycle (e.g., FDA AI SaMD guidelines). - Ideally, formulate the business and optimization problem before the data acquisition. If the data was acquired before, make sure measurement and sampling are appropriate to solve the problem. - Co-develop AI algorithm and impact guardrails in the application simultaneously to inform risk & impact assessment. - The concept of bias is applicable beyond protected information such as gender or ethnicity. Analyze systematically if your AI algorithm & app performs differently for different sub-populations that are relevant to the problem at hand. This may include feature combinations (e.g. diabetic men over 50).

One possible way to mitigate AI bias involves fine-tuning models with exceptional datasets. In the current landscape of AI, base models are largely trained on enormous datasets in an unsupervised fashion. However, these datasets often mirror societal biases, imbuing the AI models with similar prejudices. This bias might, to a large extent, be rectified through the process of fine-tuning. The challenge lies in the need for high-quality, diverse datasets to ensure the effectiveness of this adjustment. Thus, significant efforts and resources are required to generate and curate thousands of such high-caliber samples. This rigorous process could contributes to the development of more fair and equitable AI systems.

As an AI practitioner, you can use these three strategies to mitigate bias: 1. Diverse and Representative Data: Endeavour to use diverse and representative data. This means ensuring that the data you use to train your AI systems reflects the diversity of the people who will be affected by its decisions. 2. Bias Auditing: Periodically audit your AI system for bias. There are statistical methods and tools that can help detect and quantify bias in your AI models. 3. Algorithmic Fairness: Use techniques for mitigating bias in the algorithms themselves. For example, you can use different models for different subgroups if a model is found to be biased for certain groups.

Another approach is constitutional AI - where a 2nd AI reviews the output of the original AI to determine if the output aligns with the values in the constitution. There are lots of examples of "this is biased" in the training data, and it's likely that a 2nd AI can watchdog the original AI for bias.

It's also true that by reframing the problem or use case, to change how you use or respond to an AI powered inference can mean that while the system might remain biased, it does not have a negative impact on a disadvantaged group - this may be for example by giving "at risk" young offenders additional support, rather than punishing them - a positive response means that a bias does not negatively impact the target group. Indeed sometimes bias can be positive - if your system as it stands is already imbalanced, or a group has received unfair treatment in the past, you may want to bias the system toward that group to rectify this discrepancy.

Monitoring bias is state-of-the-art and yields safer and better AI applications. Automate not only monitoring but also alerting and automated model re-training where possible. 1. Monitor data & concept drift. (a) features, (b) labels/rewards, and (c) feature-label relationship. There are plenty of tools for this purpose. Monitoring labels and concept drift goes hand-in-hand with continuous label acquisition which can be expensive or slow. Plan for this early on. 2. Monitor changes in how the algorithm is used (frequency, users, downstream processes, purpose). Continue to gather feedback directly from the end user. 3. Consider putting continuous training in place (and automating it). This has emerged as state-of-the-art in many AI areas.

I whole heartly agree with the above statement. I see AI being used out there to "evaluate and Judge" people without their knowledge or consent... AI is being used incorrectly as "Judge Dread" (Judge, Jury, and Executioner)... and more alarming that the conclusions the AI is bringing fowrard are factually incorrect due to BIAS and poor methodologies. Great harm is being done to people out there! With this in mind, I propose that AI use and AI development should be a regulated professions, in the same manner that professional engineering is a regulated profession. Licensed professional engineers should supervise AI, because our engineering profession has a very strong code of ethics at its core. This ensures people are not harmed by AI.

Education also includes learning about the industry best practices to detect and mitigate bias. Some of the emerging techniques are: 1?? Synthetic data: It is not uncommon to have “gaps” in training data. Companies are using fully or partially synthetic data to overcome these data constraints 2?? Fairness Toolkits: Open source tooklits such as IBM's AI Fairness 360 or Microsoft's Fairlearn provide fairness metrics and algorithms to detect and mitigate bias across the ML pipeline 3?? Ethical Governance: Companies are establishing dedicated internal teams, such as ethics committees or model risk teams, to conduct independent reviews of AI/ML models as they’re built, aiming to prevent risks to both customers and the company

Relying on humans to cure biases that emerge from AI is counter intuitive. An algorithm that defines the markers and patterns of biases will be a more elegant way to solve this issue. We need to lean on large learning models to ingest cultural information and then feed them to machine learning algorithms.