What are the latest developments in machine learning explainability?

1 Why explainability matters

Explainability is vital not only for ethical and legal reasons, but also for practical and technical ones. For instance, explainability can validate and debug ML models by locating errors, biases, and anomalies. Additionally, it can improve and optimize ML models by uncovering new features, interactions, and insights. Moreover, it can facilitate communication and collaboration between stakeholders by providing transparent and comprehensible explanations of ML outcomes and decisions. Finally, it can build trust with users by demonstrating transparency and accountability of ML processes and results.

2 How to measure explainability

Measuring explainability is not a one-size-fits-all process, as it depends on the context, purpose, and audience of the ML model. However, there are common dimensions and criteria that can be used to evaluate explainability. These include global versus local explainability, model-agnostic versus model-specific explainability, and intrinsic versus post-hoc explainability. Global explainability refers to understanding the overall structure, parameters, and behavior of the ML model. Local explainability relates to understanding the specific input, output, or instance. Model-agnostic explainability applies to any type of ML model while model-specific explainability exploits the characteristics of a particular type of ML model. Intrinsic explainability is achieved by designing the ML model to be interpretable while post-hoc explainability uses external methods or techniques after training.

3 What are the new methods

There are numerous methods for achieving ML explainability, depending on the type, complexity, and goal of the ML model. Recently, counterfactual explanations, Shapley values, and attention mechanisms have been developed or improved. Counterfactual explanations show how the ML output or decision would change if the input or some of its features were different. Shapley values quantify the contribution of each feature to the ML output or decision, while attention mechanisms highlight the relevance or importance of each part of the input. For example, a counterfactual explanation for a loan rejection could be: "You would have been approved if your income was higher by $10,000". A Shapley value explanation for a credit score could be: "Your credit score is 700, which is composed of 300 from your payment history, 200 from your credit utilization, 100 from your credit mix, and 100 from your credit length". An attention mechanism explanation for a text summarization could be: "This is a summary of the article, which focused on the words 'climate change', 'emissions', 'targets', and 'policies'".

4 What are the new frameworks

Recently, some new frameworks have been proposed or adopted that provide a systematic and comprehensive approach to ML explainability, by integrating different methods, metrics, and principles. These include XAI, which strives to make ML models explainable, understandable, and trustworthy; LIME, which uses local surrogate models to explain the ML output; and SHAP, which uses Shapley values to explain the ML output by combining different methods like kernel, tree, or deep learning explainers. All of these frameworks aim to address four key challenges: how to produce explanations, how to evaluate explanations, how to present explanations, and how to ensure explanations are aligned with human values and expectations.

5 What are the new tools

Recently, various tools have been developed or enhanced to facilitate and automate the implementation and visualization of ML explainability. These provide user-friendly and interactive interfaces and platforms. For example, the What-If Tool enables users to explore and analyze the ML output or decision for any type of ML model, by manipulating the input features, observing the changes in the output, and comparing different scenarios and outcomes. Captum provides a unified and extensible API for ML explainability, supporting feature importance, attribution, and visualization for PyTorch models. The Explainable AI Platform integrates ML explainability with Google Cloud services, offering model insights, feature attributions, counterfactuals, and explainable monitoring for AutoML and custom models.

6 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?