5 Challenges of Explainability

Explainable artificial intelligence (XAI) is an attempt to make the result finding of nonlinear programmed systems transparent in order to avoid so-called black-box processes.

Below are five challenges in explainability to consider.

(1) Spurious correlations

• can be learned from the data, often hampering the model's ability to generalize and leading to poor real-world results.

(2) Proxy objectives

• resulting in large differences between how models perform offline, often on matching proxy metrics, compared to how they perform when deployed in the applications.

(3) Loss of debuggability and transparency

• leading to low trust as well as the inability to fix or improve the models and/or outcomes. Furthermore, this lack of transparency impedes adoption of these models, especially in regulated industries e.g. Banking & Finance or Healthcare.

(4) Loss of control?

• due to model practitioners' reduced ability to locally adjust model behavior in problematic instances.

(5) Undesirable data amplification

• reflecting biases that don't agree with our societal norms and principles.

The points on a slide

last but not least: If you are dealing with AI-Bias ...

Murat