Serverless Thumbnail Creator - Highly Scalable Event Driven Architecture in AWS

Let us say you are running a photo album application in AWS. Whenever a user uploads(event) an image to AWS S3 Bucket, you want to create thumbnails or profile picture of the images uploaded by users. We can use S3 Event notification along with lambda to do it. The high-level architecture of our thumbnail creator application is shown above.

If you are interested in followed this article as a video demonstration,

Since S3 Events and Lambda can scale very easily for a large number of users, We do not have to worry about scaling our 'thumbnail creator' app. The added bonus is that, lambda has built-in retries(defaults to 3 times) in case S3 Events is not able to successfully trigger the lambda. This feature takes care of the resiliency of our application.

Prerequisites

• AWS CLI pre-configured

Clone the repository

git clone https://github.com/miztiik/serverless-thumbnails-creator.git

Customize the deployment

1. In the serverless-thumbnails-creator directory, Edit the ./helper_scripts/deploy.sh to update your environment variables.
2. You will have to create an S3 bucket in your account and update the BUCKET_NAME in the below section

AWS_PROFILE="default"
AWS_REGION="us-east-1"
BUCKET_NAME="YOUR-BUCKET-NAME-GOES-HERE" # bucket must exist in the SAME region the deployment is taking place
SERVICE_NAME="serverless-thumbnails-creator"
TEMPLATE_NAME="${SERVICE_NAME}.yaml"
STACK_NAME="${SERVICE_NAME}"
OUTPUT_DIR="./outputs/"
PACKAGED_OUTPUT_TEMPLATE="${OUTPUT_DIR}${STACK_NAME}-packaged-template.yaml"

Save the file.

Deployment

1. We will use the deploy.sh in the helper_scripts directory to deploy our AWS SAM template

chmod +x ./helper_scripts/deploy*.sh
./helper_scripts/deploy.sh

Test Event Processor

1. Upload an object to the Source S3 Bucket created by the stack(The bucket name should be something like serverless-thumbnails-creator-srceventbucket-zwpgvaxxb3qh). You will be able to see the output in the Destination S3 Bucket(In your account *tgtbucket) three directories with resized images. In the lambda logs you will see the following output

{
  "status": "True",
  "TotalItems": {
    "Received": 1,
    "Processed": 1
  },
  "Items": [
    {
      "time": "2019-05-09T18:02:24.534Z",
      "object_owner": "AWS:AIDAUR7KWXJQLWZZ56LRA",
      "bucket_name": "serverless-thumbnails-creator-srceventbucket-zwpgvaxxb3qh",
      "key": "wt-cloudtrail-100.png"
    }
  ]
}

That is it. You can extend this functionality to,

1. You can detect faces and recommend sharing to friends & family
2. You can detect location or places and recommend creation of vacation albums
3. You can auto-correct/beautify images.
4. With the location/places information, you can recommend restaurants or other entertainment options.

The possibilities are endless. Go ahead, fork the repo and create your own value on top of it. Dont forget to send us a PR.