Making AI Explainable – Adopting a Two Part Methodology

As the interest in AI has exploded, slowly but surely the topic if explainable AI has emerged, overtaking Ethical AI in importance and relevance to thought leaders.? Given, I have done some work in this area in the past, I thought I would try to contribute something to the developing dialogue.

As a starting point, I would like to point out that one shouldn’t associate explainability with right or wrong. XAI is not about the determination of the outcome but rather about the soundness that went into it. This is why complexity is the enemy of explainability and why deep learning models, such as those that use very large neural networks, esp. via transference techniques (which create a compounding problem) are far more challenging to diagnostically explain than other types that are purely based off measurable statistical calculations.

As a result of this truth, if one conducts any cursory research on XAI, two interconnected topical domains will emerge to try to tackle explainability. The first of these will focus on the importance of linking explainability to process. By this, people are talking about establishing a framework by which one is able to unpack in an auditable way the raft of selections that data scientists need to make in order to design and implement the right types of models as well as the data wrangling processes that generate the desired outcomes. When one is creating, documenting and codifying processes, policies and procedures for this purpose, there needs to be a bridge built between the model and human actors to enable even the most complex models to be portrayed via simplified editions, (this replicates element of the LIME test btw). The framework also needs to include rollback procedures in the model test phase so that human supervisors can clearly separate and examine different decision points, and the impact of change on subsequent actions to both root out any type of human heuristics or data led biases for example, as well as fallacies in model selection or interpretation that facilitated overfitting or statistical sampling errors.? This is best accomplished by different stakeholders working together to establish the critical points of risk in each stage, and the right set of checks needed to root out the presence of decisions that could trigger the entire elimination of a certain set of outputs or reduce their presence creating new dispersions that cannot be replicated in shape or density.

There is a significant body of academic work as well as practical corporate work being built around this aspect in many ways replicating the three lines of defence models and 4+ eye decision techniques that are applied toward risk management and compliance. The pace of advancement suggests to me that software solutions to articulate, set-up and manage the framework will start to either appear as part of the overall AI operating systems or become part of the functional landscape of enterprise risk management soon enough.? In fact, I am quite sure, that on commercial grounds alone, both big tech AI leaders, as well as those partnering with them as both data providers and domain experts, are already introducing individuals with the specific tasks of tackling explainable AI in this way.

This takes me on to the second domain, where explainability, esp. in relation to more complex systems are being tackled through the use of techniques normally applied to game theory (SHAP for example). In this area, models are becoming explainable through an examination of attribution, sensitivity, and contribution of factor inputs. This approach is particularly useful to apply in scenarios where AI is making decisions because humans can’t or don’t at present. ?Certain lending scenarios related to thin credit personas seem to be particularly relevant as they allow for the potential use of non-traditional data sets (i.e., those with no proven or known influence on default or delinquency outcomes) to form a part of an underwriting decision.

While adopting the first approach in the explainability journey will be a defensible approach, a more robust additional method assigned to more complex AI systems would apply statistical techniques that can articulate the most prominent contributors, their weightings on the decision score, and their dependent and independent nature. It should be noted that since complex AI models are most desirable when they can decide at scale, as opposed to just facilitating one, using the gaming theory approach is only effective as an explanatory tool by recognizing that neural networks are neither linear nor simplistic. Thus, outputs need the ability to not only examine the raw inputs but the derived inputs that neural networks create to mimic heuristics (which humans do not know how to codify) and their subsequent contribution to risk taking, risk mitigation, accountability, and responsibility. ?First level analysis will reveal, in my experience potential anomalies in causality, but it will require further sensitivity and dependency reviews before a complex AI solution can start to resemble a white rather than black box system for examination.

Our capabilities to deliver this, if the academic research is accurate, appears to still be mathematically immature which probably explains why phantasms are still a real threat in semantic modelling, and why neural network techniques are still uncommon outside of situations where human data scientists are seeking to develop alternatives to brute form computing to optimize decision making in game theory situations.

Regardless of this situation, it is clear to me that organizations that either want to leverage open source systems or to build their own models based on proprietary data sets shouldn’t be putting the cart before the horse, so to speak, when it comes to tackling explainability. In fact, as a strategy leader, I would absolutely insist that any activities with the intention of putting AI systems in production need to be introduced around both a program of risk assessment as well as explainability.? Adopting this approach alongside the application of one or both of the topical domains will go a long way to curb reputational and regulatory damage without compromising positive business and operational objectives.

?

?