Changing Retail With Neural Networks

We use neural networks in our everyday life, experimenting with new tricks and tools to photoshop a selfie or to decipher protein structures. Nowadays, neural networks are much more than that - they’re used to solve business challenges. Let’s take a look at how we, at Eyrene, are transforming retail and auditing with 12 neural networks.

What we do

The product is an app for in-store auditing. The solution we offer helps sales representatives recognize goods and create reports, speeding up the entire process by 3-4 times (this depends on the size of the store, categories, and the number of products).

The way it works is very simple. First, a merchandizer installs the app (available on Android and iOS). Then they photograph the shelves and upload the images to the program. The neural network recognizes the product within 1-3 minutes and calculates its key performance indicators. Products are recognized correctly 95% of the time.

Eyrene enables even inexperienced employees to work effectively. Not being familiar with the client’s range of products is not an issue, as the app collects useful hints like product appearance. It reduces the time, spent on gathering data and simplifies training new employees.

How our neural networks work

Neural network #1

This network controls the image quality according to certain criteria. The picture must be of good quality and taken under the correct angle within the right distance to display the items and their price tags.

Neural network #2

The second network identifies whether the photo is real or fake. People have a tendency to fool computers in hopes of saving time. Moreover, some negligent employees might try to photograph the screen of another device, showing an image from some other store and their own. Another trick is taking a photo of merchandising books. In these unfortunate and few cases, our neural network will work to find tiny details that identify the fraud.

Neural network #3

This neural network can identify the frame orientation. If it sees an upside-down image, it understands how to rotate it. It is important that the shelves are properly fronted and faced with the products in their correct upright position.

Neural network #4

We use perspective correction, an algorithm that aligns the photo. It makes sure the horizontal lines are horizontal, and the vertical ones are vertical. This enables a more accurate processing.

Neural network #5

Neural network #5 determines the points of sale. One frame can have two points of sale in different categories, which implies different business rules and indicators. The same concept is applied if the product is placed in both a regular rack and a refrigerator. The major task here is to control the display of goods in a refrigerator, especially in a branded one.

Neural network #6

This one helps to identify objects in the frame. Moreover, it can handle both simple layouts of rectangular boxes in rows as well as non-standard layouts.

Neural network #7

This algorithm is in charge of the product and shelf recognition. The main challenge is to recognize the shelf when the products are placed in large piles.

Neural network #8

Network #8 does the price recognition. There are many price tags: handwritten, typed, from small and big retail stores. The neural network reads the price on the price tag, identifying everything from the largest fonts to the little numbers in the lower right corner.

Also, this algorithm can figure out if the price tags are mixed up. It is done by comparing the number on the price tag with other prices for the same product in the store.

Neural network #9

It determines whether the price is regular or promotional. We have a large catalog of promotional price tag samples from our clients.

Neural network #10

The most important of the neural networks determines which products are in the frame. You need to upload information about products and their variations into the system in advance. We receive this information from the client or from third-party sources.

Recognition:

• By SKU. Our task is to determine which brand it is and the category, taste, and size.
• By brand format (if it is a competitor's product). If the client is not ours, but we know this brand and what category it belongs to, it is important to locate it in order to take it into account for analytics.

Neural network #11

In all the categories we work with, there are different package sizes. They are recognized by a special algorithm that learns to correlate sizes and prices. If the price is significantly different among different sizes, the network specifies the displacement, grams, and similar KPIs.

Neural network #12

The last step is to assemble the panorama. The algorithm puts the images together after processing so that we don't count one shelf or product twice because the images were overlapped.

After that, we are ready to calculate the indicators. We count the share of the shelf, the performance of the assortment matrix, and other indicators.

The most challenging part of a project like this one is identification. Products in the same category often look very similar. We have to understand which one we are identifying. This is the reason Eyrene contains not one neural network, but a dozen.