How can you ensure explainability in a deployed ML model?

?? Prioritize Model Explainability for Trustworthy ML! I'd advocate for a proactive approach where program managers meticulously structure their ML journey around explainability. Initiate by setting concrete explainability goals tailored to your target audience. Opt for models and datasets that champion transparency and align with your objectives. Integrate renowned explainability methods like SHAP or LIME to decipher model decisions. Regularly assess the clarity and reliability of generated explanations, seeking user feedback when necessary. Lastly, comprehensively document and visualize these explanations, serving as a testament to your model's credibility.

It is not only about the goals but adapting the whole process to more explainable one. Because having explainable model (no matter how complex) should be always a goal (at least at my field - financial markets). Firstly, know your label - what it is, how is the dynamics of label data. Secondly know your data - test and understand correlations, causality, feature engineering is where a lot of explainability is hidden. Thirdly - always choose less complex model if it is not significantly worse than complex one. Fourth - use model fingerprints to explain what is happening inside, for both inputs and their combinations - computation heavy but worth it.

By understanding the specific needs and expectations of your target audience and the unique constraints of your domain, you can tailor your explainability approach to provide the right level of detail and relevance. This ensures that your explanations are not only accurate but also genuinely helpful to those who rely on your model's predictions and recommendations.

Good combination is chosen here, analyze feature importances to know which features affect the model the most. Also analyze the effects, but not only single ones but also feature pairs, triples, etc. Because your models is non-linear and not easily explained as generalized linear models. Great methodology is model fingerprints, but it is quite computationally difficult. If your model doesn't calculate feature importances itself, you can construct it yourself. Easy to calculate is the standard dev of model results when changing one variable and not touching others - do some input grid for one variable, calculate STD and repeat on some data subset - the mean of STDs is your feature influence, repeat for all features on the same data subset

To choose a method, I would say start by evaluating the specific needs & constraints of project. Selection will depend on the type of ML model has been deployed; different methods may be more suitable for decision trees, linear models, neural networks, or ensemble models. Check about who will be using the explanations - non-technical stakeholders might need simpler, high-level insights. Regulatory requirements for transparency & interpretability are crucial in context as some industries or regions have specific rules regarding model explainability. Also, understand the trade-offs between model performance & explainability, could impact accuracy. Btw, do consider the cost associated with implementing a particular explainability method

Several tools and frameworks can assist in deploying ML model explainability. Examples - - SHAP (SHapley Additive exPlanations): It is a Python library that offers a unified approach to feature importance and feature effects. It's suitable for various model types, including tree-based, linear, deep, and kernel-based models. - LIME (Local Interpretable Model-agnostic Explanations): It works well with various types of models and data, including text, images, and tabular data. - What-If Tool: It is a web-based tool that integrates with TensorFlow, which allows users to explore and compare a model's predictions and performance. It's particularly useful for examining how the model behaves with different subsets and modifications of the data.

To simplify the complex task of implementing explainability for models, various tools and frameworks are available. SHAP is a Python library that uses Shapley values from game theory to provide unified feature importance and effects explanations for diverse model types, including trees, linear, deep, and kernel-based models, and offers interactive visualizations. LIME is another Python library that creates interpretable models around individual instances using local approximation, applicable to various model types and data formats, yielding both visual and textual explanations

After generating model explanations, the next step is to evaluate and enhance them based on your explainability objectives and target audience. To gauge the explanations' quality and utility, accuracy can be assessed by comparing them with model outputs on various inputs or testing them on synthetic or adversarial examples. Consistency should be evaluated to ensure that explanations remain stable and do not contradict each other when inputs, models, or methods are slightly modified

Begin by defining clear & measurable metrics for explanation quality. These metrics could include factors like coverage, accuracy (the correctness of explanations), and consistency (how explanations align with model behavior). It's important to periodically assess these metrics to make sure explanations meet defined standards. Also, benchmarking explanations against domain-specific or industry-standard benchmarks can provide an external perspective on explanation quality. Post having areas of concern or say opportunities for improvement, an iterative process of refining explanations could be initiated. It can include revisiting ingredient used for generating explanations, improving the clarity of the language used in explanations etc.

Use Interpretable Models: Choose models that are inherently interpretable, such as decision trees or linear models, especially if the problem allows for their use. Ensemble methods like Random Forests can also provide insights into feature importance. Feature Importance Analysis: Analyze and communicate the importance of each feature in the model's decision-making process.

Well, it may begins with establishing a comprehensive tracking system to log & organize explanations, complete with version control & timestamps. Regular data & model evaluations are critical, including drift detection to spot shifts in data distribution, fairness audits to uncover bias, and model evaluation to detect errors. In case of drift, strategies for addressing it, such as retraining the model with more recent data, are necessary to keep explanations relevant. Mitigating bias is also critical; if bias is detected, work on re-engineering the features, applying fairness-aware models, or introducing post-processing steps could help eliminate unfair behavior in both model outputs and explanations.

when someone asks you about the interpretability of machine learning algorithm models. Depending on the audience, there are different approaches to take. For your boss, they generally want a simple, intuitive understanding of how the algorithm model achieves such accuracy. If it's a question from the user operations team, in addition to using similar language as with your boss, they also hope that you can uncover some key features. This can provide them with inspiration for designing subsequent operational strategies. At this point, you can use the Shapley value which is a concept borrowed from cooperative game theory. It is used to assign a value to each feature, indicating its contribution to the prediction of a specific instance.