CutMix Augmentation for Object Detection

When building a computer vision model, sometimes, we get an F1-score accuracy that is not good enough. Then, we decide to do some optimization algorithms to search for optimum hyperparameters to improve our accuracy. However, sometimes our accuracy doesn't improve significantly. We may ask, is it because of our model?

The answer is not always. Many sources can cause the model to be unable to learn from the images we give. Imbalanced class instances can be a problem. Imagine we have to develop a model that can classify multiple classes of ripeness of fruits, the proportion of unripe fruits may be larger than the proportion of ripe fruits, therefore, the number of fruits classified is imbalanced between ripe and unripe. How to solve this?

The solution is augmentation. Augmentation is a technique to reproduce images by manipulating the images. We can do an augmentation technique called "CutMix". CutMix is a technique to selectively cut the object by its bounding boxes at the pixel level, remove it from the image, and add it to another image. If we have 90 instances of class ripe and only 10 instances of class unripe, by CutMix, we can have a balanced 50-50 instances for both classes. CutMix can be done algorithmically. Below are examples of datasets after CutMix implementation on oil palm fruits on trees.

Besides, we can make use of photo editing tools such as Adobe Photoshop to do CutMix manually. We have a collection of cropped oil palm fruits with different levels of ripeness that we scraped from a website image repository.

The following process is to add these cropped objects on other images to introduce the addition of the minority class instances to our training datasets. Below are examples of images before and after CutMix implementation. We cannot almost notice the differences between the added objects and the original objects. To make good quality CutMix images, we need to try several editing parameters such as image blending to blend the objects with the background to make them look real.

There is a significant improvement in Precision, Recall, and Mean Average Precision at 50% Confidence Level (mAP50) between training without augmented images with CutMix and training with augmented images with CutMix. Below is a summary of the results.

We can see that CutMix augmentation increases the performance of the model to better generalize on minority class instances.

Yohanes Nuwara - Founder & CEO of RadiaX AI Consulting

To see more of our services, visit our website: https://radiax.godaddysites.com/