AIOps: Interpretability using Disjunctive Normal Form

Arun Ayachitula & Upendra Sharma

Interpretability and Explainability of AI/ML models have been differentiated in the literature. Some machine learning models are easy to explain as they could be a simple combination of raw (un-transformed) input variables – for instance, linear regression, decision trees, etc. But when the problem at hand is complex, either because the number of raw features is large (for instance, thousands or even millions) or the problem cannot be easily solved without transforming the features to some new set of features. Another common problem is the massive dataset size; in such problems, the performance of the machine learning models is also critical. Real-world problems often translate into complex learning problems, and advanced machine learning methods and models are designed to solve complex learning problems while keeping performance high. Often such algorithms transform the raw input variables so that the learning objectives (namely accuracy, generalization, and speed/performance) can be improved. This makes them hard to interpret/understand, i.e., why the model’s output is the way it is.

In real-life business problems, it becomes essential to explain the models to the business owners or decision-makers to improve confidence in the model and the end user's trust. An explanation can be considered a humanly comprehendible representation of the model’s decision, which is approximate but accurate.

Our specific use case is to classify a ticket based on its description. We use a linear classifier that uses a large number of mono and bigrams to classify a ticket. It is often not apparent from the classification the reason/intuition of the classification decision, especially in the case of misclassification. A natural choice for such explanations is rules expressed in a Disjunctive Normal Form, i.e., disjunctions of conjunctions of literals, for instance?

Where p, q, r are sentence literals and V etc. are logical operators, namely, disjunction (logical OR), conjunction (logical AND), and negation, respectively. A Disjunctive normal from the representation of a logical rule/formula is essentially a disjunction of conjunction clauses (a conjunction clause is a sequence of conjunctions of literals).

In our case, the literals are the pertinent positives of our input variables. The presence of a literal means that the feature is present in the description, while the negation of a literal/word means that the feature should not be present in the description. For instance, below is a DNF rule (explanation) of a classification done by a linear classifier on the ticket description.

This means the ticket got classified as a filesystem ticket because it had words like diskInodeUtilization, filesystem, and mounted and did not have the word database in its description.

The essence of our approach is first to train a linear classifier like Passive Aggressive classifier on our training data, using ticket descriptions and their class labels. We carefully evaluated all the classifications and misclassifications of many tickets from our training and test data and evolved a set of Disjunctive Normal Form (DNF) rules for each class/label. The DNF rules use the pertinent positives that seem to define the class and use them to extract the classification rules.

Explainability & Interpretability: AI formats, such as decision rule sets, decision trees, and decision tables, are simple enough for people to understand. Supervised learning of these models is directly interpretable. We use a mix of Explainability, computed by identifying the pertinent positive features from a given ticket using various AI/ML techniques, and Interpretability using the disjunctive normal form (DNF). The DNF rule features are typically the Pertinent Positives from the tickets and are highlighted below in the ticket description.

Incident ticket summary: N1VL-PA-APB169_Guest File System:/var|Partition Utilization_VirtualMachine ae81283e-dbac-4bc4-b780-bd37b07d3446/One or more virtual machine guest file systems are running out of?disk?space

Work in progress:?Exploring the use of pertinent negatives as negations in the rule extracted as a disjunctive normal form.

Acknowledgements

Sincere thanks! to Rohit Khandekar for his contributions to the DNF implementation in AIOps.

Further Reading:

[1] Explainable AI: https://cloud.google.com/explainable-ai

[2] Explainable AI (XAI): https://www.ibm.com/watson/explainable-ai

[3] Interpretability versus Explainability: https://docs.aws.amazon.com/whitepapers/latest/model-explainability-aws-ai-ml/interpretability-versus-explainability.html

[4] https://ai.googleblog.com/2022/01/controlling-neural-networks-with-rule.html -- Neurips-2021

[5] https://docs.aws.amazon.com/whitepapers/latest/model-explainability-aws-ai-ml/interpretability-versus-explainability.html

[6] Cohen, William. (2000). Fast Effective Rule Induction. Twelfth International Conference on Machine Learning: 1995. 95. 10.1016/B978-1-55860-377-6.50023-2.