Automate image recognition with AWS Machine Learning services

Many businesses and organisations require real time image processing for various use-cases such as Searchable Image Library, Face-Based User Verification, Sentiment Analysis, Facial Recognition, Image Moderation and many more

Before the introduction of Amazon Rekognition, organisations followed the usual approach of employing a person or third-party costly solution, but today with the aid of this new technology, all image processing can be done without any human touch.

Amazon Rekognition Custom Labels feature allows users to create their own customised machine learning (ML)-based image analysis capabilities to discover unique items and situations relevant to their specific use case. Customers may utilise Amazon Rekognition Custom Labels to get state-of-the-art performance for their particular image analysis needs instead of having to train a model from scratch, which requires specialised machine learning knowledge and millions of high-quality labelled photos. Customers in the insurance industry may use AWS AI services to automate the document processing pipeline for claims processing.

Solution overview

Below diagram represents end to end solution that we typically see in an image processing solution. We will walk through each parts of the application, starting from serverless web application deployment, on-demand image processing using AWS ML Services and generating insight for the same data using Amazon QuickSight.

The solution built using the following key services:

Amazon Rekognition is a service from AWS that allows you to perform image analysis using machine learning.

AWS Lambda?for highly scalable functions to perform image analysis using Amazon recognition programmatically.

Amazon S3?— for a cost-effective data backup and object storage solution for analytics, artificial intelligence, machine learning, and web applications.

Amazon DynamoDB?– Key-value and document database for solutions that require rapid response times at any scale.

Amazon QuickSight - business analytics service that we will use to generate Insights on summary data gathered through Amazon Rekognization.

Amazon Simple Email Service?(Amazon SES) – service for email sending, we will utilize same for share summary data for specific image to configured emailed.?

?Prerequisites

For this walkthrough, you should have the following prerequisites:

• An?AWS account?with?SES Setup
• AWS?user ?with permissions to deploy the?AWS CloudFormation?template.
• AWS IAM role for Lambda function having access for DynamoDB, SES and Rekognition.
• Valid email IDs to receive email notifications.

Phase1–Microservice and web applications: Enterprises can utilize serverless stack to deploy web applications built using microservices and modern user interfaces, without having to worry about sizing or predicting capacity. This Serverless Microservices and web Application will be utilized by end users/partners to upload document which require on-demand processing. More details are available @?https://aws.amazon.com/getting-started/hands-on/host-static-website/

Phase2- Automate Image Processing using AWS Lambda, Amazon Rekognition and save summary in Dynamo DB and send mail to Administrator/Business User

In the image processing phase, we extract label such as ‘Person’, ‘Water’, ‘Sand’, ‘Palm Tree’, and ‘Swimwear’ (objects), ‘Beach’ (scene), and ‘Outdoors’ (concept) from image via Amazon Rekognization using event driven architecture build using Lambda functions.

Walkthrough: Create a Lambda function with the console?using step mentioned here with Python and paste the below source code and click on deploy. This sample will tell you whether image is cat or not.

Note:

-??Please update AWS Region with the region where your SES is setup.

-???Add S3 as a trigger point for Lambda function where you will upload image from S3.

-???Add below environment variables.

§?SUMMARYTABLE: <<DynamoDB Table>>

§??SENDER : <<email ID configured in SES>>

§???RECIPIENT:?<<Recipient email ID’s>>

import json

import os

import boto3

import urllib.parse

from botocore.exceptions import ClientError

def lambda_handler(event, context):

??? bucket = event['Records'][0]['s3']['bucket']['name']

??? print(bucket)

??? photo = urllib.parse.unquote_plus(event['Records'][0]['s3']['object']['key'], encoding='utf-8')

??? print('photo:', photo)

??? client=boto3.client('rekognition')

??? summaryTable = os.environ[‘SUMMARYTABLE]

??? sender = os.environ[‘SENDER’]

??? recipient = os.environ[‘RECIPIENT]

?

?

?

??? response = client.detect_labels(Image={'S3Object':{'Bucket':bucket,'Name':photo}},

??????? MaxLabels=2)

?

??? responsetext = photo + ' is not a Cat'

??? for label in response['Labels']:

??????? print ("Label: " + label['Name'])

??????? print ("Confidence: " + str(label['Confidence']))

??????? print ("Instances:")

??????? if label['Name'] == 'Cat' and label['Confidence'] > 80:

???????????

??????????? responsetext = photo + ' is a Cat'

?? ?????else:

??????????? print(photo +'Label is not cat')

???????????

???????????

??????? for instance in label['Instances']:

??????????? print ("? Bounding box")

??????????? print ("??? Top: " + str(instance['BoundingBox']['Top']))

??????????? print ("?? ?Left: " + str(instance['BoundingBox']['Left']))

??????????? print ("??? Width: " +? str(instance['BoundingBox']['Width']))

??????????? print ("??? Height: " +? str(instance['BoundingBox']['Height']))

??????????? print ("? Confidence: " + str(instance['Confidence']))

??????????? print()

?

??????? print ("Parents:")

??????? for parent in label['Parents']:

???????

???

??? AWS_REGION = <<AWS region>>

??? CONFIGURATION_SET = "ConfigSet"

??? SUBJECT = "CAT Test"

??? BODY_TEXT = ("Amazon SES Test (Python)\r\n"

???????????? "This email was sent with Amazon SES using the "

???????????? "AWS SDK for Python (Boto)."

??????????? )

???????????

??? CHARSET = "UTF-8"

???

??? # Create a new SES resource and specify a region.

??? sesclient = boto3.client('ses',region_name=AWS_REGION)

???

??? # Try to send the email.

??? try:

? ??????#Provide the contents of the email.

??????? response = sesclient.send_email(

??????????? Destination={

??????????????? 'ToAddresses': [

??????????????????? RECIPIENT,

??????????????? ],

??? ????????},

??????????? Message={

??????????????? 'Body': {

??????????????????

??????????????????? 'Text': {

??????????????????????? 'Charset': CHARSET,

??????????????????????? 'Data': responsetext,

??????????????????? },

??????????????? },

?????????????? ?'Subject': {

??????????????????? 'Charset': CHARSET,

??????????????????? 'Data': SUBJECT,

??????????????? },

??????????? },

??????????? Source=SENDER,

??????????? # If you are not using a configuration set, comment or delete the

??????????? # following line

??????? )

??? # Display an error if something goes wrong.??????????????

??? except ClientError as e:

??????? print(e.response['Error']['Message'])

??? else:

???? ???print("Email sent! Message ID:"),

??????? print(response['MessageId'])n        

Phase3: Get Insight from processed image

In the stage we can generate various dashboards using Amazon QuickSight on top of the data saved in DynamoDB.

Key business choices are efficiently made depending on the quality of accessible data. As a result, the capacity to evaluate and generate relevant insights from processed data would aid in a clearer knowledge of the status and goals. Amazon QuickSight is used to improve data quality by adding a presentation layer, resulting in meaningful information and, ultimately, successful decision-making.

Consideration

Amazon Rekognization is a high-scalable packed with easy-to-use APIs for processing image. It is equipped to learn from new data and continuously upgrades itself by adding new features.

Conclusion

In this blog, we showcased the common challenges in image processing, and how we can use AWS ML services to automate image processing including insight generation.

Amazon recognition is not limited to this specific use case only but we can utilize same for lots of other use cases such as

-??Searching across your library of image content with text keywords

-??Confirm user identities by comparing live images with reference ones

-???Analyze trends based on public images including the sentiments and emotions of the people in the images

-???Detect explicit and suggestive content and automatically filter it for your purposes retrieve text from images