To teach a computer: computer vision training and COVID scans (Part 2 of 2)

In the previous article, we looked at the way that AI can scan x-ray images and identify potential COVID patients. The technology is useful and can help make hospitals safer. However, there are a few challenges with this approach. Namely, the AI platform has to learn what COVID looks like.

Challenges

AI-based screening for COVID patients appears to be a great solution, but it presents its own challenges. Computer vision is a very data-intensive application of AI. In order to train the platform, it needs to be fed thousands of specific, labeled images. Each batch of new images trains the platform on the features, shapes, or outline of specific objects. In the COVID example, this can be mucus deposits, or damage-points in the lungs consistent with other COVID images. The success rate is approximately 94%, meaning that only 6% of patients need a second opinion (Hao et al. 2021).

Labeling the images is where the issue arises. The labeling process may appear simple at first glance. It isn’t difficult to take a series of pictures of cats or dogs and place a square to identify the animal in question. But lung scans? A doctor would need to review and analyze the picture before it can be labeled correctly. Ideally, you would use more than one doctor to avoid human error as well, up to 6 doctors are necessary to make sure each image is as accurate as possible. This means that 6 highly-trained experts need to spend valuable time labeling these images.

Worse still is the volume of images. To train an AI platform to be able to scan for COVID damage in the lungs, it needs to review more than 14,000 images of healthy, COVID, and pneumonia patients. This is a huge volume to process, especially for six doctors. Assuming a conservative 30 seconds of time spent per image, this volume can take up to 600 hours to process.

The data’s availability is another problem. A person’s X-ray scans are private medical information, protected in most jurisdictions by privacy laws. Patients would have to consent to their information being used to train the AI. This makes it difficult to obtain enough samples to be able to train the platform.

These two issues means that despite the volume of data needed to train the AI, there is a limited availability for it.

Labeling data automatically

Samsung SDS has been researching a means of automating this labeling process through neural networks. In essence, by training a secondary AI with the images, allowing it to automatically label future images of the 14000, it could make the data screening process far more efficient. The network needs to be trained on a minimal number of images. Samsung’s AI scientists have studied this problem and has identified a means of training the network with far fewer manually labeled images.

The team applied different methodologies and tested various ways of arranging the data inputs, until they came upon a solution to the problem. Instead of feeding all the images, why not apply deep learning to teach the network to analyze and label the rest of the images itself?

The idea is similar to training a student to perform, say, long division in school. By giving them a long list of problems of increasing difficulty with the answers penned in, the student will eventually learn the principles enough to write a test. But rather than that long and tiresome process. A better method for teaching complex problems is to start from somewhere. Teaching the basic principles with sample problems is a critical first step, followed by assignments and quizzes. By checking the problems that the students get wrong, or have trouble answering, then the teacher can identify which problems stand to provide the newest information to students. By working through these, then assigning more practice, students can learn faster and more efficiently. This process can be applied to any learning problem, and it certainly can be applied to AI.

First, instead of having six doctors label all 14,000 images, the machine is fed the image library whose features have been studied and processed by the AI through unsupervised feature learning. This process provides a baseline, or a “supervised classifier” for the AI to go from for the next series of images. Then, after this first step, the machine learning algorithm rank-orders the library of images. The order is based on which images the AI finds the most “confusing”, or rather, which images provide new information that the AI does not recognize. The top 200 images are then labeled and fed back to the AI to teach as much new information as possible. This process can then be repeated with the next 200 confusing images. Eventually, the accuracy climbs and each new image provides a diminishing amount of new data to the AI. When the AI is deemed “educated” enough, then the labeling stops.

Samsung found that through this new training procedure, they were able to train the network to label lung x-rays with only 10% of them being manually identified. Every input into the network after this point gave fewer and fewer returns, until it was virtually able to label all of the x-rays with 95% accuracy. This made it more accurate than the fully manual labeling process. Moreover, close to 93% accuracy was obtained at only 10% of the images used (Hao et al. 2021). This meant that the network could label, and therefore train a computer vision platform to make use of these x-ray scans, much faster.

The remaining images still need to be checked for accuracy, of course, but this takes much less time than labeling. Instead of 30 seconds per image, it would take around 5 seconds per image. This means that after the initial 1400 labels (which will take 70 hours), the remainder can be checked in much less time (about 87 hours). This reduces the number of hours required for labeling, which lowers the cost and increases the efficiency of both the network platform and the data used to train it.

What’s next?

Samsung’s Autolabel technology can train visual analytics platforms and computer vision to analyze complex images. The COVID use case presents some of its own challenges and opportunities, but Autolabel allows for more data efficiency in working through this large volume of images.

Bibliography

Hao, Heng, Sima Didari, Jae Oh Woo, Hankyu Moon, and Patrick Bangert. 2021. “Highly E?cient Representation and Active Learning Framework for Imbalanced Data and Its Application to COVID-19 X-Ray Classi?cation,” 16.

Credits: This article was written by Ryan Cann.