AI is lying to you? (Part 1)

How can you trust a model if you cannot understand how it reaches its conclusions?

Unboxing the Black Box

In the last decade, Artificial Intelligence (AI) systems' performance is reaching, and sometimes surpassing, the human level on many tasks.

AI-based technologies are increasingly being used in our daily life; think about movie recommendations of Netflix, friend suggestions on Facebook, neural machine translation of Google, or speech recognition of Amazon Alexa. Not only, but AI-enabled computer applications are also being increasingly used in three important fields of our lives, health, and finance.

Deep Neural Networks (DNNs) demonstrate great success in learning complex patterns that enable them to make predictions about new data, overcoming traditional machine learning algorithms' performance when working with high-dimensional data and complicated functions. However, high performances of DNNs come at the cost of low explainability.

How and why they reach the predictions is unknown.

It is complicated and often impossible to get insight into the networks' internal workings and explain their learning processes and final results in a human-understandable way. This causes them to be called black-box algorithms.

The lack of transparency represents a barrier to adopting these systems for those tasks where interpretability is essential, like autonomous driving or medical applications. Delegating black-box models without the possibility of an interpretation may be critical and can create discrimination issues.

The possibility of understanding why “Computers say no!” is an essential element of trust, and in the context of deep learning, this is increasingly being recognized by regulators.

In May 2018, the European General Data Protection Regulation (GDPR), in Article 22, established the right of explanation:

“The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her.”

How to overcome?

Read my full article on Medium: AI is lying to you? (Part 1)