How do you use tools and frameworks for explainable AI?

Adapting the level of explanation is quite like customizing a dashboard for different roles within an organization. The C-suite might want a high-level view of how an AI system impacts business metrics. Whereas a data scientist may require a detailed look into the model’s intricacies. As Albert Einstein once said, "If you can't explain it simply, you don't understand it well enough." Simplification isn't about dumbing down; it's about clarity, and that's paramount in AI and applicable across audiences and roles.

There is already great work in this space with tool such as tensorboard and Whatif, Whatif is a tool i've used in Google that can be used to visualize the impact of changes to the input data on the output of an ML model. These are all great indicators, we need to do more, in particular where you have a huge corpse of datasets and they are being trained using DNNs

Feedback loops are the backbone of any iterative process, and AI is no different. It is critical to design human-in-the-loop to ensure we augment machine's insights with human intelligence. In a computer vision solution we built for drug discovery, the AI-driven biological cell detection and counting interface was designed for periodic human audit. Any misclassification could be easily flagged and commented for the right cell type and count. This two-way street enhanced the model's performance over time and simultaneously educated the data science team on business nuances they weren't aware of.

This is a major part of the ML workflow, and hence why you have MLOps, to incorporate feedback loop, to keep the model fresh, the ML model needs to be treated as a constant iterative cycle which needs to be watered and fed with fresh data, training etc and this is where enteprrises struggle the most as it is not easy to have a end to end automated pipelines which can upkeep the state of the ml systems

In my opinion, learning from firsthand experiences is crucial in the realm of XAI. While theories and models offer a foundational understanding, real-world scenarios unveil complexity and unpredictability. Facing unanticipated situations, adapting to unforeseen variables, and navigating unexpected challenges deepen insights into how explainability truly functions in practice. Delving into actual cases, be they successes or failures, provides invaluable perspectives, highlighting nuances that might be overlooked in simulations or controlled settings. In essence, while theory lights the path, practical experiences refine our ability to navigate the often unpredictable landscape of XAI.

Make sure you learn about the main packages that are used in explainable AI. In Python, these are LIME, SHAP and `interpret`. A lot of people also use XGBoost importance ranks. Most people that are asked to implement some level of explainability on their projects will end up using one of these packages, and will be unaware of the assumptions and limitations of each one. If you familiarize yourself with them, you can be sure that you are using them correctly: that your data meets the required assumptions, that you correctly interpret the output and that you know the limitations.