Anomaly Detection and Interpretability 2020

"There is no wealth like knowledge, and no poverty like ignorance"

       - Buddha

"Share your knowledge. It's a way to achieve immortality"

                       - Dalai Lama 

Cloudera Fast Forward labs publish reports along with a prototype on emerging technologies. Recently they open-sourced two of their reports - Deep learning for Anomaly Detection and Interpretability 2020. Both reports are well researched and written in an engaging manner. 

Deep learning for Anomaly detection report covers applications of Anomaly detection, why semi-supervised learning is suitable for anomaly detection, how variational autoencoders can help in detecting anomalies, how businesses can avoid huge labelling costs by using semi-supervised learning, models suitable for low latency and temporal use cases, A prototype, landscape of tools and ethics of anomaly detection.

I particularly liked the flow diagram which explains how to decide the right modelling approach for varied business use cases. Finally, I liked the conclusion when the report says, "Different is not necessarily bad".

The second report, Interpretability 2020 covers the power of interpretability. I liked the use case of pneumonia. A hospital trains a model to predict the "probability of death" of patients suffering from pneumonia. One of the features of the model is "Has Asthma". The model predicts a (Image Credit: medium.com)

low probability of death for patients suffering from Asthma. A Pneumonia patient suffering from Asthma is subjected to high risk and receives excellent care during treatment. This excellent care is the reason for their higher prognosis than average. If this model is deployed into production then it will be deadly. It includes a feature "Has Asthma" which should not be included in the final model. This is known as leakage. Interpretability of the model will help in such cases by providing information about why such predictions are returned.

The report covers about why interpretability is required, what are white-box models, some white-box models like SLIMs, GAMs and Rule lists, interpretability of black-box models using Perturbation technique by LIME and Game theory-based Shapley values. The write up is backed by a prototype, landscape of packages of interpretability and Ethics and regulation. The report also includes an excellent science fiction story based on interpretability (Highly recommended).

The reports are extremely readable, well written and worth investing your time. The Cloudera website says, "Moving forward, all new reports will be publicly available and free to download. In addition, we will be providing access to updated versions of older reports over time, so check back often to explore available free research". #cloudera when can we expect to read your other Fast forward labs reports? we are eagerly awaiting.

What are you reading presently? Did you come across any machine learning or Artificial intelligence reports, blogs or tutorials worth sharing? Please share!