Explainable AI
Introduction
AI systems, often operate like black boxes: we send the inputs and receive outputs, but we don't see the internal decision-making process. How can we trust these outputs?
Thats where Explainable AI comes to rescue. Purpose of Explainable AI is to open this black box, providing insights into the model’s inner workings.
Explainability vs Accuracy Tradeoff
Basic AI models are often more explainable but less precise for eg a Linear Model , while complex models like Neural Networks are highly precise but harder to interpret. This precision in complex models comes from their ability to represent complex real world patterns and scenarios.
Let's take a real world example and understand what happens behind the scene.
Imagine a dataset containing information for candidates appearing for Graduate School Admissions , including test scores, university rating, statement of purpose , letters of recommendation and cumulative grade point average. Using this dataset we want to predict graduate school acceptance.
We will use two different types of Models : A Simple Decision Tree Classifier vs A Complex Neural Network for predicting the same thing.
As shown below a simple Decision Tree Classifier generates a score of 0.79 and also explains the rules how it arrived to that prediction.
Rules clearly represents how DecisionTreeClassifier predicted the value 0.79. Unfortunately such rules explanation cannot be generated for the Neural Network.
A complex Neural Network would generate a score of 0.90
The MLPClassifier shows better accuracy than DecisionTreeClassifier. But we can extract the Learned Rules for DecisionTreeClassifier
Lets look at certain techniques using which we can unlock the effect and bias of a certain feature on the overall prediction. These techniques can be classified under two categories. Model Specific and Model Agnostic.
Model Specific Techniques are simpler.
Check examples for Model Specific Technique
But I would like to focus on Model Agnostic Techniques to determine bias of a particular feature.
Introduction to SHAP and LIME
SHAP stands for Shapely Additive Explanation is a widely popular Model Agnostic Technique that explains the contribution of individual features in a given machine learning model prediction.
SHAP is based on game theory principles. shapley values, aim to fairly distribute the "payout" among players based on their contribution to the total outcome. In machine learning, SHAP values quantify how much each feature contributes to the prediction.
SHAP Explainer
Neural networks can be very accurate, but understanding their decisions can be challenging due to complexity. SHAP Kernel Explainer is a general explainer that can derive SHAP values from any neural networks.
Let’s look at how to apply the Kernel Explainer in a real world use case.
领英推荐
Kernel Explainer
Let's use Kernel Explainers to analyze two neural network models, one for regression, and one for classification.
The first model is an MLPClassifier named mlp_clf, trained to predict the risk of heart disease in patients based on features such as chest pain type and heart rate.
The second model is an MLPRegressor named mlp_reg, trained on the insurance dataset to predict the charges a person pays based on features such as smoking status and age.
To compute feature importance, I will aggregate the SHAP values by calculating the mean absolute values across all samples. For classification, I select the index 1. Plotting the results, I see that for insurance charges, age and smoking status are the most important, while for heart disease, chest pain type and thalassemia are the most influential features, which aligns well with insurance and medical industry insights.
LIME
LIME, which stands for Local Interpretable Model-agnostic Explanations, is specifically designed to explain the predictions of complex machine learning models. As the name suggests, this technique offers understandable explanations on a local, per-instance basis, making it agnostic to the type of model being analyzed.
Let's explore how to use LIME to explain two models trained on the graduate admissions dataset. A regressor that predicts the likelihood of admission and a classifier that determines whether a student will be admitted. While here we examine both models for easier comparison, in real-life applications we would choose one based on the specific data and question at hand.
The explanation.as_pyplot_figure() method generates visual representations of explanations. Both regression and classification models show that higher GRE and TOEFL scores increase admission chances, while lower SOP, University Rating, and CGPA scores decrease them.
Contrasting this with SHAP, LIME's local explanations show the specific conditions and ranges that detail how features influence an individual prediction, enhancing transparency and trust in model predictions at a granular level. In AI, this level of insight is crucial for ensuring that complex models can be trusted.
Conclusion
Hope you enjoyed this post. Now you have learned about local explainability, where you applied SHAP and LIME to individual data points, enhancing your ability to provide detailed explanations for individual samples on different kinds of datasets.Both SHAP (SHapley Additive Explanations) and LIME (Local Interpretable Model-Agnostic Explanations) are valuable tools for explaining complex machine learning models, but they excel in different aspects: SHAP provides a more comprehensive, global understanding of feature importance across the entire dataset, while LIME focuses on detailed, local explanations for individual predictions, making the choice between them dependent on the specific need for either broad model insights or granular prediction analysis within a given context.