Robots in Retail

1. Retail is Complicated

Here is a model of a retail enterprise.? It shows the large number of interfaces that must be managed for a retailer to succeed.? As one can see each application has numerous interfaces to other applications all over the retailer’s world. To work in a retail environment, robots must interact with many of these interfaces.? Let’s explore several use cases.

2.OMG/Retail Robot Interfaces

Here are the UnifiedPOS V1.16 (https://OMG.org/Retail) published set of robot interfaces for use in Retail:

Video Capture Interface

Video capture device class has the following capabilities:

? ???????? Get the captured frame data.

? ???????? Take a photograph and record it in a file.

? ???????? Take a movie and record it in a file.

? ???????? Read the encoded data from the bar code label.

? ???????? Detect objects such as faces.

?Individual Recognition Interface

The Individual Recognition has the following set of capabilities:

·??????? Analyzes the image of the camera and recognizes Individual s such as people and balls.

?Sound Recorder Interface

The Sound Recorder has the following capability:

? ???????? Save the recorded sound to a file.

Voice Recognition Interface

The Voice Recognition has the following capability:

? ???????? Convert spoken words to strings.

? ???????? Types of voice recognition

? ???????? Voice recognition is mainly a method of specifying word candidates to be recognized and waiting for those words.

? ???????? There are the following four types of voice recognition.

o?? Yes/No/Cancel recognition

§? It listens to the sound of words classified as Yes / No / Cancel defined by the device.

§? For example, the voice “OK." is classified as Yes.

o?? Word recognition

§? The application specifies a list of words and listens for the voice of that word.

o?? Sentence recognition

§? The application specifies a word and a list of patterns of the sentences using it and awaits the sound of the sentence.

o?? Free recognition

§? Voice recognition left to the device is performed without specifying the word to wait.

?Sound Player Interface

The Sound Player has the following capability:

·??????? Play audio file.

?Speech Synthesis Interface

The Speech Synthesis has the following capability:

·??????? Convert text to speech and speak.

?Gesture Control Interface

The Gesture Control has the following capability:

·??????? It controls the operation of various joints.

·??????? The operation is automatically controlled by interlocking various joints and other devices.

·??????? Register and play the defined pose and motion.

Automatic control Interface

Automatic control of a joint means to automatically control a joint on the device side, such as tracking according to the movement of a person's face, in cooperation with a camera or the like connected to the device.

Pose / Motion

·??????? Pose refers to setting the position of one or more defined joints.? For example, it is an action that lifts a hand.

·??????? To execute a pose, specify the pose in the start Pose method or the pose name defined in the device.

·??????? To execute motion, specify the motion file name or the motion name defined in the device in the start Motion method.

Device Monitor Interface

The Device Monitor Device has the following capability:

·??????? Get values measured by various devices.

·??????? Notify the application of changes in values measured by various devices.

Graphic Display Interface

The Graphic Display has the following capability:

·??????? Displays the specified image.

·??????? Play the specified movie.

·??????? Display the specified web page.

·??????? Notify the application of changes in the load status of the web page.

3.Retail - Robot Application Interaction

Robots in retail require interacting with a wide number of different retail systems.? Each system will have its own set of messages to properly communicate with the robot.? OMG/ARTS has message sets to support almost every system in a retailer’s environment.

This is a sample of the retail systems that must interact with robots:

·??????? Robot Interfaces – UnifiedPOS Robot Schemas – used to manage the robot.

·??????? Inventory Interfaces – Inventory Schema – used to identify if the items are in stock.

·??????? Price Interfaces – Price Schema – used to calculate the prices in the shopping basket.

·??????? PCM Interfaces – PCM Schema – used to verify the right product is being ordered.

·??????? POSLog Interfaces – POSLog Schema – used to calculate the amount in the shopping basket.

·??????? Stocking Interfaces – used to help locate the items on the shopping list.

·??????? Smart Assistants – used to help figure out alternative items.

·??????? Stock Locator Interfaces – used to identify the location of the items.

·??????? Location Interfaces – Location Schema – used to help efficiently drive the robot around the store.

·??????? Buying Interfaces – used to reorder items.

·??????? Item Interfaces – Item Schema – used along with PCM to verify the right items are being ordered.

·??????? Customer Interfaces – Customer Schema – used to identify the customer for this order.? This includes things like promotions, home address, legal age for buying alcohol, etc.

4.Pick, Pack, Deliver Customer Order Use Case

These are some of the interfaces necessary to support this use case - Robot Interfaces, Inventory Interfaces, POSLog Interfaces, IoT Interfaces, Location Interfaces

It starts with the Refrigerator.? The refrigerator identifies items to be added to the customer’s shopping list.? This uses the IoT (Internet of Things) interface.

Next we need a Shopping List.? This is recorded in the POSLog Message Interface.? POSLog is one of the most popular messages in the entire inventory of retail standards.? (I led its creation – I am the Wizard of POS),

When the customer is ready the Shopping List gets sent to the store

Next we need an Inventory Message to help us locate the items on the Shopping List.

We need to have a Shopping Cart.? That is another use for the POSLog Message.? In this case we add in location information gathered from the Inventory Message and Location Messages.?

With that information we can now build a path for the Robot to follow to pick these Items. We are now ready to send that path off to the Robot.

Of course, we now have to drive and control the Robot itself using the UnifiedPOS Robot Commands.

Once the physical shopping cart is filled, we need to be able to pay for the items.? This is where the set of Payment Messages come into play.

The last step is to put the purchased items in the delivery method to deliver them to the customer. The robot takes the customer’s purchase and places it in either a drone, self-driving automobile or the customer’s car for delivery to the customer.

These are the various standard interface messages needed to carry out this use case:

·??????? IoT Message – talking to the device

·??????? POSLog Message –

o?? Building the Shopping List

o?? Filling out the Shopping Cart

o?? Sending the receipt to the Customer.? This uses the Digital Receipt which is a subset of POSLog.

·??????? Inventory Message – Identifying what is in inventory

·??????? Payment Messages – paying for the purchase

·??????? Location Message – locating the items in the physical store.

·??????? Robot Messages – controlling the movement of the robot around the store

5.Audit Shelf Use Case

Shelf-audit- to aid in inventory management and price compliance, these Robots autonomously scan store shelfs to determine that the correct price is displayed, that merchandise is placed in the correct location, and to call out missing shelf-facings (see Picture X).?

It starts with the Robot being able to travel around the store without running into customers, things in the aisles, walls, end caps, etc.? Doing this requires knowing the layout of the store and the current location of the robot.?

In addition, it must have the vision to see dynamic things that can appear in its path.? This is where the Video Capture capability comes into play.

The Video Analytics capability is simultaneously used to look at the shelf.? This is where things get more complicated.?

The easy part is knowing what items should be at this location on the shelf.? That takes having access to the Inventory with the Inventory Management Message, the Item Message, the Product Content Message and the Location Message.? These four in combination tells one what is on the shelf at this location.?

As it counts the items at a particular location, it can identify and report if the quantity falls below the capacity threshold.?? This helps keep customers happy.? Nothing makes a customer more frustrated than looking for an item and not being able to find it because the store is out of that item.? If this happens too often, the customer will go to a competitor’s store.

One side note, customers regularly place items wherever they are when they decide they don’t want a particular item.? That requires employee’s pickup those misplaced items and put them back in their correct location.? Robots can help locate these items and communicate that information to the nearest employee.? This helps make your store look better and your customers feel better being in your store.

As the robot moves around the store, it can also identify any problems like spills.? It can then immediately report these problems to the employees for immediate correction.? They can even hang around to protect customers until the problem is resolved.

Once the item is located a check on making sure the correct price is on the shelf label using the Price Message.

If it is a food item, one can verify the item is still safe by using the Traceability Message.

These are the various standard interface messages needed to carry out this use case:

·??????? Way Finding Message – develop path for movement through the store

·??????? Robot Interface – control movement of the robot

·??????? Price Message – evaluate the correct price is on the shelf label

·??????? Location Message – locate the shelf and place on the shelf for a particular item(s)

·??????? Item Maintenance Message – identify the item

·??????? Inventory Management Message – help locate the item in the store

·??????? FIM (Traceability) Message – verify the item is still prior to its expiration date

·??????? Product Content Management Message – picture piece of the item information

·??????? Video Analytics Message – visual identify the various situations, such as spills, item on the shelf, etc.

6.Autonomous Floor Cleaner Use Case

The Cleaning Robot travels around the store cleaning the floor.? This first requires knowing the layout of the store.? Then as moving around, it has to look for obstructions and people.? It then makes adjustments.

These are the standard interface messages that are necessary to clean the floors:

·??????? Way Finding Message – develop path for movement through the store

·??????? Robot Interface – control movement of the robot

·??????? Location Message – locate the shelf and endcaps

·??????? Video Analytics Message – visual identify various situations, such as customers, dropped items on the floor. Etc.

7.Spill/Hazard Use Case

Spill/Hazard Use Case - Spill/Hazard detection- bots that circulate a store to detect spills and other hazards and alert employees for clean-up.

1.?????? Receive Notification of a Spill on Aisle 3 - A customer spilled her soft drink on aisle 3

2.?????? Travel through the Aisles and Detect Spill – one way is to compare the current image against the base original image of the area.

3.?????? Notify Associates about the Spill

4.?????? Move to the Spill

5.?????? Clean up the Spill

6.?????? Release the Robot to Continue around Store

7.?????? Post Incident Report of Spill

These are the various standard interface messages needed to carry out this use case:

·??????? Robot Messages – UnifiedPOS Robot Schemas – used to manage the robot.

·??????? Product Content Management Messages – PCM Schema – used to verify the spill

·??????? Location Message – used to help efficiently drive the robot around the store.

·??????? Planogram Message – Helps drive the robot around the store.

·??????? Wayfinding Message – Helps drive the robot around the store.

·??????? Retail Operational Data Model (ODM) – Stores information about the store layout and what the prime images of the store to help identify a spill.

·??????? Video Analytics Interface –

o?? Object of interest (Hazard type) contains area of interest

o?? In query – request Object of Interest (Spill) and response returns area of interest.

8. Delivery Management Use Case

1.?????? Get the Online Order

2.?????? Relate the Customer to the Order

3.?????? Send the Robot to pick and pack the items

4.?????? Identify the Customer’s Deliver Service

o?? Use:

§? Drone

§? Postal

§? Autonomous Car

§? Delivery Robot

5.?????? Deliver the purchase to the Customer

These are the various standard interface messages needed to carry out this use case:

·??????? Robot Interface Messages

·??????? Retail Operational Data Model (ODM) – Customer Information

·??????? Location Interface Message

·??????? Warehouse Pick, Pack and Ship Interface Messages

·??????? Autonomous Car Interface

·??????? APDS Parking Standard Model

9.Warehouse Pick, Pack and Ship Use Case

1.?????? Receive Packages at the dock

2.?????? Unload Packages from the trucks

3.?????? Sort Packages for storage in the warehouse

4.?????? Transport Packages to Shelves

5.?????? Store Packages

6.?????? Pick Packages following the picklist

7.?????? Move these picked packages to packaging stations

8.?????? Audit Packages to verify the package is correct

9.?????? Deliver Packages to the appropriate store/customer

These are the various standard interface messages needed to carry out this use case:

·??????? Robot Interface Messages

·??????? Retail Operational Data Model (ODM) – Customer Information

·??????? Customer Message

·??????? Item Maintenance Message

·??????? Location Message

·??????? Stocking Message

·??????? Stock Locator Message

·??????? Warehouse Pick, Pack and Ship Interface Messages

10. Picking Groceries in Warehouse Use Case

1.?????? Receive the picklist at the warehouse

2.?????? Layout the route to pick the items.? This may involve merging multiple picklists to optimize the robot’s movement.

3.?????? Move the robot around the warehouse following the picklist.

4.?????? Pick the item(s) off the shelf

5.?????? Pack the item(s) for delivery and move to the loading dock.

These are the various standard interface messages needed to carry out this use case:

·??????? POSLog Message – communicates the customer’s shopping list

·??????? Inventory Management Message – identifies what’s in inventory

·??????? Stock Locator Messages – tells where the items are located within the warehouse

·??????? Stocking Messages – tells where the items are located within the shelfs

·??????? Location Message – helps define the wayfinding message to move the robot around the warehouse

·??????? Item Management – Identifies the UPC code on the item

·??????? Product Content Management Message – visual of the individual item used to help verify the correct item is selected.

·??????? Video Analytics Message – visually identifies the item on the shelf.? In addition if automate conveyors are used, it can be used to make sure the box is not tilted on the conveyor belt and about to fall off.

·??????? Picking Message – records picking a particular item from the picklist

·??????? Robot Messages – moves the robot around the warehouse following the way finding path derived from the picklist.

11.; Conclusion

As you can see, there is a lot that has to be done on the retail side to interact with these robots in some intellectual way.

To see more details check out this YouTube Video - Robot in Retail (youtube.com)

To find out all the retail message standards, check them out at https://www.OMG.org/Retail.? There is somewhere around 20,000 attributes contained in all this work.? If you are not using them, you are wasting company resources by recreating the wheel.? On top of that, you have to create new interfaces with every company you have to connect with.? More waste of company resources that make you no money.

Here is an overview of all these standards:

The Blueprints, Design Documents, Technical Reports and Best Practices documents are no longer available, but you can get them by reaching out to me.? There is an enormous amount of great information contained in this work.? It is an entire encyclopedia of Retail Technology.

-?????? To further find out how to use all this incredible work, check out my book "ARTS for Retail Using Technology to turn your consumers into customers and make a profit” (3 book series) Kindle Edition https://lnkd.in/g4SqtRB8. You get a MILLION dollars worth of knowledge from over 1500 subject matter experts for less than $25.

-??????In addition, I’ve created a whole set of YouTube videos https://www.youtube.com/user/richardthegeek/videos to show how to use this information.

-??????I’ve also created a School of Retail with courses:

·?????? on the "Modern Retail Architecture" It is a 10 lesson self-paced course.

·?????? on "Unified Commerce" https://schoolofretail.thinkific.com/courses/unified-commerce..

·?????? on "Modern POS World" It is a 14 lesson self-paced course.

·?????? on "Standard Agile Retail Data Model - Who did what"

-?????????? In addition, I’ve posted out 6 articles on various aspects of retail technology -???????Security in Retail: https://lnkd.in/ga9sN6_q

-???????What is a Blockchain? https://lnkd.in/gciyHkX

-???????Artificial Intelligence in Retail - https://lnkd.in/g2Y8kpaF enjoy

-???????AI Business Process Model https://lnkd.in/gMZSTFFw

-???????Evolving Retail - https://lnkd.in/gtVVk3mW

-????????Create an Omnichannel or BOPIS Presence - Create an Omnichannel or BOPIS Presence | LinkedIn

Reach out to me to learn how to get access to these courses.? “May the force be with you!”

?