Pattern Tracking - Machine Learning

Over the last few years, the computing power of GPU’s has increased rapidly enabling us to compute complex operations in parallel at high speed. As a result with the use of Big Data, Machine Learning and especially the use of convolutional neural networks is becoming increasingly important.

Idenitfing objects in complex images plays a fundamental role in Machine Learning and is already used in many ways to optimise processes. During my year at the EPFL I had the chance to participate in many interesting courses about Machine Learning. In this article I want to show the process of modifying an existing state of the art Machine Learning algorithm, named openpifpaf, to identify predefined patterns on images. This model can then be used for a "followMe" mode on a robot. The project is based on the EPFL course "Data and Artificial Intelligence for Transportation".

The first step of modifying an existing Machine Learning algorithm is to analyse its function and outputs. In this case the openpifpaf architecture outputs a multi-person 2D human pose estimation which is shown below. Therefore in our next step we want to change the output to an object detection task.

In our second step we dive into the data input structure of the given architecture. There we can observe that the architecture is based on keypoint detection. That means the algorithms takes keypoints such as ”left-shoulder” or ”right leg” as an input and tries to learn these specific points for multiple objects on an image. The information about the keypoints and many other parameters are given through the OpenSource Coco Dataset. Since we do not need 15 keypoints (left shoulder, right leg, etc.) for each object, we reduce the number of keypoints per object to one. This single keypoint is located in the center of the object and indicate its class (Person, Car, etc.).

If you just need an object detector you are now done. But since our goal is to design a model that detects a very unique pattern we need to do one other modification. Instead of using the objects in the Database, we will create our own pattern and paste this image randomly on the images of the dataset. As input for the architecture we will again only use the location of the center of the image that we pasted. In a final step we just let this algorithm run for around 40.000 images and around 5 epochs and we will get very good results.

As a result we built in a few steps a very well performing model that is able to track a predefined pattern in an unknown environment. In this case we did not even need to create an own architecture but instead use a given OpenSource state of the art architecture. For more projects and a more detailed report please check my GitHub (https://github.com/yklose).