Face Analysis

Introduction

AWS Rekognition is a cloud-based service that provides highly accurate and scalable face analysis. Leveraging deep learning algorithms, it can identify, analyze, and compare faces within images and videos. The service enables developers to incorporate powerful facial recognition capabilities into their applications without the need for extensive machine learning expertise.

Key Features of AWS Rekognition:

a. Face Detection: Rekognition can detect faces within images and videos, providing information about the position and size of each face.

b. Facial Analysis: It goes beyond detection and provides detailed insights about facial attributes such as age range, gender, emotions, and even the presence of facial hair.

c. Face Comparison: Rekognition allows for face-to-face comparison, making it useful for applications like identity verification, access control, and authentication.

d. Face Search: With face search, developers can create face libraries and search for specific individuals within large collections of images or videos.

e. Celebrity Recognition: Rekognition can identify celebrities within images, enabling features like automated tagging and content personalization.

PoC

In this PoC, I have used AWS Rekognition’s detect_faces API to get facial expressions from the uploaded photo. Rekognition service can detect one or more faces in the uploaded photo. The recognition service detects various attributes of a person just by analysing the photo. It detects attributes like gender of a person – male or female, age of a person, facial expressions like weather person is happy, sad, whether person is smiling etc. It also captures emotions of a person like he is Angry or surprised or sad etc. ?See the output section for detailed response.

Detection of facial expressions can play a vital role in Video communication tools. Few cases that I can think of are: a teacher can detect attention span of students during online class session OR an agent can understand interest of customer in their product / speech during a video call. Using facial analysis data can create greater possibilities to lead your conversation. ?

I found it to very easy to use Rekoginition service for detection of facial analysis. It requires knowledge of creation of S3 bucket, adding a lambda function that consumes Rekognition api and parsing the JSON response received from Rekognition service. See below sample code and the output:

Lambda Function Code

import json
import boto3

client = boto3.client ("rekognition")

def lambda_handler(event, context):
??? bucket_name = "sct-images"???
??? file_name = "index/image1.jpg"
??? response = client.detect_faces(Image={'S3Object':{'Bucket':bucket_name, 'Name': file_name}},Attributes=['ALL'])
    for face in response['FaceDetails']:
       print(json.dumps(face))
        

?

? Output from detect_faces()

Input Image

Output from AWS Rekognition

{

??????????????"BoundingBox": {

?????????????????????????????"Width": 0.22321808338165283,

?????????????????????????????"Height": 0.13971003890037537,

?????????????????????????????"Left": 0.4043685793876648,

?????????????????????????????"Top": 0.30275967717170715

??????????????},

??????????????"AgeRange": {

?????????????????????????????"Low": 42,

?????????????????????????????"High": 50

??????????????},

??????????????"Smile": {

?????????????????????????????"Value": true,

?????????????????????????????"Confidence": 96.3752670288086

??????????????},

??????????????"Eyeglasses": {

?????????????????????????????"Value": false,

?????????????????????????????"Confidence": 97.7378158569336

??????????????},

??????????????"Sunglasses": {

?????????????????????????????"Value": false,

?????????????????????????????"Confidence": 99.99673461914062

??????????????},

??????????????"Gender": {

?????????????????????????????"Value": "Male",

?????????????????????????????"Confidence": 99.97653198242188

??????????????},

??????????????"Beard": {

?????????????????????????????"Value": false,

?????????????????????????????"Confidence": 94.51205444335938

??????????????},

??????????????"Mustache": {

?????????????????????????????"Value": false,

?????????????????????????????"Confidence": 98.37017059326172

??????????????},

??????????????"EyesOpen": {

?????????????????????????????"Value": false,

?????????????????????????????"Confidence": 99.97492218017578

??????????????},

??????????????"MouthOpen": {

?????????????????????????????"Value": true,

?????????????????????????????"Confidence": 95.36556243896484

??????????????},

??????????????"Emotions": [{

?????????????????????????????"Type": "HAPPY",

?????????????????????????????"Confidence": 99.64119720458984

??????????????}, {

?????????????????????????????"Type": "SURPRISED",

?????????????????????????????"Confidence": 6.270044326782227

??????????????}, {

?????????????????????????????"Type": "FEAR",

?????????????????????????????"Confidence": 5.898842811584473

??????????????}, {

?????????????????????????????"Type": "SAD",

?????????????????????????????"Confidence": 2.1600725650787354

??????????????}, {

?????????????????????????????"Type": "CONFUSED",

?????????????????????????????"Confidence": 0.062218692153692245

??????????????}, {

?????????????????????????????"Type": "DISGUSTED",

?????????????????????????????"Confidence": 0.06220677122473717

??????????????}, {

?????????????????????????????"Type": "ANGRY",

?????????????????????????????"Confidence": 0.05336402729153633

??????????????}, {

?????????????????????????????"Type": "CALM",

?????????????????????????????"Confidence": 0.03595223277807236

??????????????}],

??????????????"Landmarks": [{

?????????????????????????????"Type": "eyeLeft",

?????????????????????????????"X": 0.48033538460731506,

?????????????????????????????"Y": 0.35637974739074707

??????????????}, {

?????????????????????????????"Type": "eyeRight",

?????????????????????????????"X": 0.5743960738182068,

?????????????????????????????"Y": 0.35905277729034424

??????????????}, {

?????????????????????????????"Type": "mouthLeft",

?????????????????????????????"X": 0.48334038257598877,

?????????????????????????????"Y": 0.40148213505744934

??????????????}, {

?????????????????????????????"Type": "mouthRight",

?????????????????????????????"X": 0.5617623329162598,

?????????????????????????????"Y": 0.40362977981567383

??????????????}, {

?????????????????????????????"Type": "nose",

?????????????????????????????"X": 0.5345738530158997,

?????????????????????????????"Y": 0.37948113679885864

??????????????}, {

?????????????????????????????"Type": "leftEyeBrowLeft",

?????????????????????????????"X": 0.44260337948799133,

?????????????????????????????"Y": 0.34564223885536194

??????????????}, {

?????????????????????????????"Type": "leftEyeBrowRight",

?????????????????????????????"X": 0.5046613812446594,

?????????????????????????????"Y": 0.34269657731056213

??????????????}, {

?????????????????????????????"Type": "leftEyeBrowUp",

?????????????????????????????"X": 0.47584980726242065,

?????????????????????????????"Y": 0.33994555473327637

??????????????}, {

?????????????????????????????"Type": "rightEyeBrowLeft",

?????????????????????????????"X": 0.5584703683853149,

?????????????????????????????"Y": 0.34426039457321167

??????????????}, {

?????????????????????????????"Type": "rightEyeBrowRight",

?????????????????????????????"X": 0.6059441566467285,

?????????????????????????????"Y": 0.3502267599105835

??????????????}, {

?????????????????????????????"Type": "rightEyeBrowUp",

?????????????????????????????"X": 0.5841835141181946,

?????????????????????????????"Y": 0.3430534303188324

??????????????}, {

?????????????????????????????"Type": "leftEyeLeft",

?????????????????????????????"X": 0.4622293710708618,

?????????????????????????????"Y": 0.35596439242362976

??????????????}, {

?????????????????????????????"Type": "leftEyeRight",

?????????????????????????????"X": 0.4986546039581299,

?????????????????????????????"Y": 0.35728761553764343

??????????????}, {

?????????????????????????????"Type": "leftEyeUp",

?????????????????????????????"X": 0.48073193430900574,

?????????????????????????????"Y": 0.3539760410785675

??????????????}, {

?????????????????????????????"Type": "leftEyeDown",

?????????????????????????????"X": 0.4804258644580841,

?????????????????????????????"Y": 0.35835638642311096

??????????????}, {

?????????????????????????????"Type": "rightEyeLeft",

?????????????????????????????"X": 0.5554451942443848,

?????????????????????????????"Y": 0.3589087426662445

??????????????}, {

?????????????????????????????"Type": "rightEyeRight",

?????????????????????????????"X": 0.5895764827728271,

?????????????????????????????"Y": 0.3595370650291443

??????????????}, {

?????????????????????????????"Type": "rightEyeUp",

?????????????????????????????"X": 0.5751330852508545,

?????????????????????????????"Y": 0.3566540479660034

??????????????}, {

?????????????????????????????"Type": "rightEyeDown",

?????????????????????????????"X": 0.5735118389129639,

?????????????????????????????"Y": 0.3609658181667328

??????????????}, {

?????????????????????????????"Type": "noseLeft",

?????????????????????????????"X": 0.5098012089729309,

?????????????????????????????"Y": 0.38528427481651306

??????????????}, {

?????????????????????????????"Type": "noseRight",

?????????????????????????????"X": 0.5445082187652588,

?????????????????????????????"Y": 0.3862284719944

??????????????}, {

?????????????????????????????"Type": "mouthUp",

?????????????????????????????"X": 0.5268710851669312,

?????????????????????????????"Y": 0.39624473452568054

??????????????}, {

?????????????????????????????"Type": "mouthDown",

?????????????????????????????"X": 0.5238800644874573,

?????????????????????????????"Y": 0.4100845754146576

??????????????}, {

?????????????????????????????"Type": "leftPupil",

?????????????????????????????"X": 0.48033538460731506,

?????????????????????????????"Y": 0.35637974739074707

??????????????}, {

?????????????????????????????"Type": "rightPupil",

?????????????????????????????"X": 0.5743960738182068,

?????????????????????????????"Y": 0.35905277729034424

??????????????}, {

?????????????????????????????"Type": "upperJawlineLeft",

?????????????????????????????"X": 0.40866947174072266,

?????????????????????????????"Y": 0.35863757133483887

??????????????}, {

?????????????????????????????"Type": "midJawlineLeft",

?????????????????????????????"X": 0.4244271218776703,

?????????????????????????????"Y": 0.40759000182151794

??????????????}, {

?????????????????????????????"Type": "chinBottom",

?????????????????????????????"X": 0.5167960524559021,

?????????????????????????????"Y": 0.4343530237674713

??????????????}, {

?????????????????????????????"Type": "midJawlineRight",

?????????????????????????????"X": 0.5903977155685425,

?????????????????????????????"Y": 0.41210123896598816

??????????????}, {

?????????????????????????????"Type": "upperJawlineRight",

?????????????????????????????"X": 0.6134564876556396,

?????????????????????????????"Y": 0.36436891555786133

??????????????}],

??????????????"Pose": {

?????????????????????????????"Roll": 3.621122121810913,

?????????????????????????????"Yaw": 7.095423698425293,

?????????????????????????????"Pitch": 8.19347095489502

??????????????},

??????????????"Quality": {

?????????????????????????????"Brightness": 91.78880310058594,

?????????????????????????????"Sharpness": 86.86019134521484

??????????????},

??????????????"Confidence": 99.99840545654297

}