Computer Vision: Feature Matching with OpenCV

Computer vision is a field of study which aims at gaining a deep understanding from digital images or videos. Combined with AI and ML techniques, today many industries are investing in researches and solutions of computer vision. Namely, think about the security procedures in the Airport: when you have to exhibit your passport, it is now common to find a camera which is able to determine whether your face is the same of that on your passport (which you have to put on a document reader). Basically, the reader is able to detect relevant features on your face (which are able to describe its shape, expression and so forth) and then match them with those detected by the camera. If the matching procedure returns a high score, it means that, with high probability, the two faces are referring to the same person.

This procedure is called feature matching, and it is the topic we are going to discuss throughout this article. For this purpose, I will use OpenCV (Open Source Computer Vision Library) which is an open source computer vision and machine learning software library and easy to import in Python.

The idea of feature matching is the following: we start with a train image and ask our algorithm (which we have to pick in advance) to find some keypoints, which are the indicators of the location of each feature. Furthermore, the algorithm will also look for descriptors associated with keypoints, which are arrays of numbers which describe the corresponding feature. So, each feature will have an associated couple of values, (keypoint, descriptor). Then we repeat the same procedure with the query image (that where we want to look for similarities): finally, we look for similar descriptors between the two images, which will indicate the presence of a match among features.

To better understand this procedure, we need to provide the answers to two questions:

• How can we state when two descriptors are 'similar'?
• How can I choose the algorithm to perform this task?

The answer to the first question is straightforward: we use the mathematical concept of 'similarity', which is distance. But how do we compute that distance? Well, it depends on the algorithm we are going to choose, so we need to answer the second question first.

There are many algorithms available for this purpose, however in this article, I'm going to dwell on ORB (Oriented FAST and Rotated BRIEF) since, differently from other candidates (SIFT and SURF) is not patented and free to use. ORB combines the functionality of two further algorithms:

• FAST (Features from Accelerated Segment Test): it is used to find keypoints
• BRIEF(Binary Robust Independent Elementary Features): it is used to find descriptors

So for ORB algorithm, namely, it is commonly used the Hamming distance as measurement of similarity, and the latter is the one we are going to use.

Now, let's see a practical application of what we've been discussing so far. My idea is selecting as input image the logo of a brand, H&M, and then try to detect the same logo in a different picture, where it is not so evident. These are the images I'm going to use:

Furthermore, I'm going to use the Brute Force (BF) feature matching as a procedure. It is a simple technique to decide which feature in the query image is best matched with that in the train image. This perfect match is elected looking at the smallest distance among those computed among one feature in the train pic and all the features in the query pic.

Let's see how it works:

import numpy as np
import cv2
from matplotlib import pyplot as plt


img1 = cv2.imread("query.jpeg",0) # queryImage
img2 = cv2.imread("train.jpeg",0) # trainImage


# Initiate ORB detector

orb = cv2.ORB_create()


# find the keypoints and descriptors with ORB

kp1, des1 = orb.detectAndCompute(img1,None)
kp2, des2 = orb.detectAndCompute(img2,None)

For the sake of clarity, let's see how the FAST component works in finding keypoints on the train image:

img_kp = cv2.drawKeypoints(img2, kp2, 0, color=(0,255,0), flags=0)
plt.imshow(img_kp)

Now let's create and plot the matches:

#create BFMatcher object

bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)


#Match descriptors

matches = bf.match(des1,des2)


#Sort them in the order of their distance.

matches = sorted(matches, key = lambda x:x.distance)

# Draw first 10 matches

img3 = cv2.drawMatches(img1,kp1,img2,kp2,matches[:10],None, flags=2)

plt.imshow(img3),plt.show()

If we increase the number of matches we want to display, we obtain the following:

#Draw first 30 matches.
img3 = cv2.drawMatches(img1,kp1,img2,kp2,matches[:30],None, flags=2)
plt.imshow(img3),plt.show()

As you can see, except for some imprecisions, this algorithm did its job.

Besides features matching, OpenCV offers a variety of tools for computer vision which are extremely powerful and intuitive to use. If you are interested in implementing that software in your Python environment, I strongly recommend you to refer to this guide.