When Deep Learning Mistakes a Dishwasher with an Oven

“Gareth, the dishwasher seems to be clogged. Please fix it.”

“Affirmative sir, will check and repair.”

“Great, I’ll be out on the deck.”

***

It’s a beautiful day in 2030 and I’m relaxing with my entertainment system. I sit up suddenly.

Sniff, Sniff. “Gareth, are you cooking something? I smell something burning.”

“Negative, sir, just flushing the dishwasher hose, sir.”

“It really smells like gas in here, let me see…. Oh no, Gareth, what did you do? That’s the oven hose you’ve disconnected, not the dishwasher. Gas is leaking…FIRE! FIRE!” 

Deep learning has achieved high levels of accuracy in recent years due in part to sophisticated advances in computer vision technologies, specifically in the area of object recognition. Since 2015, object recognition has attained an error rate of 3.5% – even lower than the 5% baseline human rate – meaning that today’s machines surpass humans in the ability to recognize objects. However, sometimes even the remaining 3.5% of errors can be critical.

Object recognition errors: Silly or serious?

Neural network mistakes can often be funny. For example, the Multimodal Recurrent Neural Network proposed in 2015 by Karpathy and Fei-Fei famously mistook a toothbrush for a baseball bat, and wrongly identified a soccer game for a tennis match. Other errors can be dangerous – even the smallest visual error made by an autonomous car or a robotic doctor can be disastrous. When it comes to a virtual technician, errors in object recognition can be humorous such as when the virtual technician mistakes a cable for a snake, but can also be destructive. Hardware devices can be ruined, software can be damaged, and dangerous situations such as electrocution can occur.

There are several reasons for neural network errors. Computer algorithms lack human common sense, therefore, the machine may fail to deduce certain logic, such as that a baby cannot lift a heavy baseball bat. Lack of computer learning is another reason. If an algorithm was not trained with sufficient data, or more simply, did not see enough images of babies, toothbrushes or baseball bats, the correct objects will not be detected.

For this reason, deep learning modules often display an accuracy level – a number which represents the percentage of successful recognition – for its ability to correctly match the object to an existing class, or category of items in which the network was trained. For example, animals, food, hardware, road signs, etc. 

How can the accuracy of the future virtual technician be enhanced to reduce costly mistakes?

Data is the key

The best method of increasing the machine’s accuracy level is by extensive data collection. For a virtual technician to accurately recognize the exact models of a wide range of devices, cables or ports – hundreds of thousands of labeled images of each of these items are required.

However, collecting and tagging masses of data can be a long, costly and painful process for an enterprise. One of the most efficient methods to execute this activity is via crowd-sourcing. Imagine that a company has several thousands of customer service agents, and that each of them captures 2-3 images during every technical support call, tagging each image with the device model and specific technical issue. In a very short time, the enterprise will build up a massive data set that can be used to train algorithms to achieve high accuracy levels.

Continued optimization

Once the algorithms achieve an acceptable level of accuracy, the enterprise must begin to focus on ensuring ongoing optimization of their algorithms. This can be done through small-scale testing. When an agent captures an image taken from customer’s smartphone, the virtual assistant can be given an opportunity to recognize the device and diagnose the customer’s technical issue. The agent then confirms or corrects the diagnosis, and in this way, allows for continuous learning and further improves the accuracy of the algorithms over time.  

In today’s age of smart homes and plethora of digital devices on the market, achieving high accuracy levels is essential for the effectiveness of the future virtual technician. Over time, with ongoing acquisition of data, continuous learning and optimization, computers will no longer mistake a cable for a snake, a toothbrush for a baseball bat, or an oven for a dishwasher… and virtual technicians will be trusted with a wider range of technical support operations.

About TechSee

Techsee transforms the customer support domain with a visual assistance platform powered by AI & Augmented Reality. We apply deep learning computer vision algorithms to learn from every customer interaction and automate the support process over time. By creating the largest data repository in the world of visual tech issues, we are able to identify technical devices and their models in up to 95% accuracy and associate them with common issues and their resolutions.