“Do you see what I see?”: Taking enterprise computer vision to edge

Until November 2019, much of the IBM PowerAI Vision (PAIV) capabilities were focused around training (Unique NVLink, Large Model Support, Distributed Deep learning …etc.) and ease of use. Now PAIV has extended it’s reach to the edge with a well thought out offering: the announcement of IBM Visual Inspector (IVI). I did some light reading over the weekend and learned about IVI. It is a native iOS/iPadOS application complementary to PAIV (https://www.ibm.com/sg-en/marketplace/ibm-powerai-vision) and geared towards enterprise users at edge. IVI uses the models trained on PAIV and performs inferencing using the integrated camera on an iOS/iPadOS device and the CoreML* models stored locally in the edge devices.

Anyone with access to Apple AppStore can download IVI from app store (https://apps.apple.com/sg/app/ibm-visual-inspector/id1486600972) as a free app and it’s ready to go for a demo with two preinstalled inference models. The demo models that are part of the downloaded App and do not require access to PowerAI Vision (i.e. connection to server side).

 IVI capabilities:

1. Users can gather data (in 'collection' mode) and perform inferencing (in 'inspect' mode)
2. Inferencing can be done in disconnected (coreML model is stored locally in the mobile device) or connected (data will be uploaded to server) modes
3. Remote management of connected devices and CoreML models
4. The device can be placed in a ‘kiosk mode’ where the end user can access only the IVI in the mobile device.

What I like:

1. Inferencing is possible in connected and disconnected mode. The disconnected mode is very handy when a user (e.g. an insurance agent visiting a disaster zone, a healthcare worker visiting a remote village) have to visit areas with no or limited communication.
2. IVI has individual authorizations for classes devices and models (e.g. person A can have authorization to model X while person B can have authorization to access models X and Y ...etc.)
3. You can push the annotated data back to server for new model development or improving existing models
4. When in a connected mode; the app will send data back to server. If app loses connection to server, it will cache the data to be sent back at a later time.

Use cases

Here are some ideas:

1. Manufacturing: Quality inspection in various manufacturing industries. Use of AI based inferencing in this context will deliver higher quality products faster through feedback and alerts. This workflow is also a way to continuously improve accuracy.
2. Healthcare: Capturing images related to a symptom or treatment in remote areas where communication is a challenge.
3. Financial services: Insurance claims inspections (e.g. A field worker inspecting hurricane damage)
4. Construction industry: inspection of buildings and surroundings.

I guess at the end of the day, the CoreML models have to be within the recommended sizes for use cases to be executed (see notes) at the edge.

Requirements, Licenses:

• Access to an on-premises or cloud instance of PAIV v1.1.5. Production use with PAIV requires a license for PAIV.
• An Apple mobile device running iOS v13.1, or later. A license is required for each mobile device on which IVI will be used when in production.

Further reading:

1. Article by Scott Soutter: https://www.dhirubhai.net/pulse/ibm-visual-inspector-computer-vision-ai-inference-your-scott-soutter/

2. Announcement of IVI: https://www.ibm.com/downloads/cas/EM-ENUSZP19-0592-CA

3. IBM Knowledge Center: https://www.ibm.com/support/knowledgecenter/SSRU69_1.1.5/base/vision_inspector.html

Next step:

Seeing is believing. If you haven’t seen IVI in action, please get in touch with me and I will be happy to share more details and/or show a demo.

What use case do you want to explore?

Notes:

• Video streaming is not supported in the initial release.
• As I understand an object detection model in the edge device can be maximum of 60MB and classification models can be up to 10 MB in size.
• * CoreML is a machine learning framework introduced by Apple. CoreML provides ready-to-use models that you can integrate into your iOS apps.