Deploy a Python Flask App to ECS Fargate
Deploy a flask backend app within the ECS Fargate
AWS Fargate is a technology that you can use with Amazon ECS to run containers without having to manage servers or clusters of Amazon EC2 instances. Since I had a chance to work with Fargate closer in AWS Cloud Project Bootcamp (weeks 6-7) in this post, I will show you a demo on how to write a task definition file for a flask app and deploying to ECS Fargate. Let’s dive right in!
What is AWS Cloud Project Bootcamp
Cloud Project Bootcamp helps students who have acquired associate-level knowledge and are at a point where they realize they need a cloud project on their resume, to progress their career goals. It combines multiple cloud services to emulate a real-world production workload.
Source-Code/Github Repo
Our cloud project is the Cruddur App and you can find the source code here. It is the starting codebase that we use in the AWS Cloud Project Bootcamp 2023. ?Since I'm in week 6 right now, I've implemented Pattern Scripts for crud operations of conversations.
Before you begin
You need AWS cli installed. In this tutorial I am going to use `create-cluster` command. However if you prefer `ecs-cli` you can use my repo as a guide or this script.
You should also export the following variables to authenticate with AWS API. Make sure to change values before exporting.
export AWS_SECRET_ACCESS_KEY="
export AWS_ACCESS_KEY_ID=""
export AWS_DEFAULT_REGION=""
export ECR_PYTHON_URL="$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/cruddur-python""
What is ECR_PYTHON_URL?
In previous weeks, I pulled python:3.10-slim-buster from docker hub for builds. To eliminate single point of failure, I tagged the python image properly and pushed to my private ECR named cruddur-python. I started referencing it as a base-python image in Dockerfile from week 6.
To sum up, ECR_PYTHON_URL is the URI of my private repo storing the base python image.
Step 1-Create CloudWatch Log Group
aws logs create-log-group --log-group-name cruddu
aws logs put-retention-policy --log-group-name cruddur --retention-in-days 1r
Step 2-Configuring Policies for Task Definition
passing sensitive data
Run the following commands to pass the values to task definition securely
aws ssm put-parameter --type "SecureString" --name "/cruddur/backend-flask/AWS_ACCESS_KEY_ID" --value $AWS_ACCESS_KEY_I
aws ssm put-parameter --type "SecureString" --name "/cruddur/backend-flask/AWS_SECRET_ACCESS_KEY" --value $AWS_SECRET_ACCESS_KEY
aws ssm put-parameter --type "SecureString" --name "/cruddur/backend-flask/CONNECTION_URL" --value $PROD_CONNECTION_URLD
assume task execution role
Firstly, I created a task execution role to authenticate with the api before interacting with Fargate. Created a file named service-assume-role-execution-policy.json:
? "Version":"2012-10-17",
? "Statement":[{
? ? ? "Action":["sts:AssumeRole"],
? ? ? "Effect":"Allow",
? ? ? "Principal":{
? ? ? ? "Service":["ecs-tasks.amazonaws.com"]
? ? }}]
}{
Then assumed the task execution role:
aws iam create-role
?--role-name CruddurServiceExecutionRole ?\
?--assume-role-policy-document file://aws/policies/service-assume-role-execution-policy.json\
assign AmazonECSTaskExecutionRolePolicy to ecsTaskExecution role
Now it was time to configure permissions with Cloudwatch and ECR. I created a json file with the following policies. Make sure to replace with your resource arn.
? "Version": "2012-10-17",
? "Statement": [
? ? {
? ? ? "Sid": "VisualEditor0",
? ? ? "Effect": "Allow",
? ? ? "Action": [
? ? ? ? "ecr:GetAuthorizationToken",
? ? ? ? "ecr:BatchCheckLayerAvailability",
? ? ? ? "ecr:GetDownloadUrlForLayer",
? ? ? ? "ecr:BatchGetImage",
? ? ? ? "logs:CreateLogStream",
? ? ? ? "logs:PutLogEvents"
? ? ? ],
? ? ? "Resource": "*"
? ? },
? ? {
? ? ? "Sid": "VisualEditor1",
? ? ? "Effect": "Allow",
? ? ? "Action": [
? ? ? ? "ssm:GetParameters",
? ? ? ? "ssm:GetParameter"
? ? ? ],
? ? ? "Resource": "arn:aws:ssm:us-east-1:023175542133:parameter/cruddur/backend-flask/*"
? ? }
? ]
}{
I assigned the AmazonECSTaskExecutionRolePolicy to the role I created.
aws iam attach-role-policy --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
--role-name CruddurServiceExecutionRole\
Step 1-Create ECS Cluster
Create a cluster named `cruddur` and set a default Service Connect namespace as `cruddur`.
aws ecs create-cluster
--cluster-name cruddur \
?--service-connect-defaults namespace=cruddur\
The above command will not cost us anything because there's no compute right now.
Step 2- Login to ECR
Login to ECR with the following command:
aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com"
Step 3- Creating Service prerequisites
1- Export the VPC ID
Store and export the value of default vpc id to `DEFAULT_VPC_ID` variable:
export DEFAULT_VPC_ID=$(aws ec2 describe-vpcs
--filters "Name=isDefault, Values=true" \
--query "Vpcs[].VpcId" \
--output text)
echo $DEFAULT_VPC_I\
2- Create a security group named crud-srv-sg
Create and export a security group
export CRUD_SERVICE_SG=$(aws ec2 create-security-group
--group-name "crud-srv-sg" \
--description "Security group for Cruddur services on ECS" \
--vpc-id $DEFAULT_VPC_ID \
--query "GroupId" --output text)\
3- Export the Subnet IDs
export DEFAULT_SUBNET_IDS=$(aws ec2 describe-subnets ?
?--filters Name=vpc-id,Values=$DEFAULT_VPC_ID \
?--query 'Subnets[*].SubnetId' \
?--output json | jq -r 'join(",")')
echo $DEFAULT_SUBNET_IDS\
领英推荐
4-Create TaskRole
aws iam create-role
? ? --role-name CruddurTaskRole \
? ? --assume-role-policy-document "{
? \"Version\":\"2012-10-17\",
? \"Statement\":[{
? ? \"Action\":[\"sts:AssumeRole\"],
? ? \"Effect\":\"Allow\",
? ? \"Principal\":{
? ? ? \"Service\":[\"ecs-tasks.amazonaws.com\"]
? ? }
? }]
}"\
aws iam create-role
? ? --role-name CruddurTaskRole \
? ? --assume-role-policy-document "{
? \"Version\":\"2012-10-17\",
? \"Statement\":[{
? ? \"Action\":[\"sts:AssumeRole\"],
? ? \"Effect\":\"Allow\",
? ? \"Principal\":{
? ? ? \"Service\":[\"ecs-tasks.amazonaws.com\"]
? ? }
? }]
}"\
aws iam put-role-policy
? --policy-name SSMAccessPolicy \
? --role-name CruddurTaskRole \
? --policy-document "{
? \"Version\":\"2012-10-17\",
? \"Statement\":[{
? ? \"Action\":[
? ? ? \"ssmmessages:CreateControlChannel\",
? ? ? \"ssmmessages:CreateDataChannel\",
? ? ? \"ssmmessages:OpenControlChannel\",
? ? ? \"ssmmessages:OpenDataChannel\"
? ? ],
? ? \"Effect\":\"Allow\",
? ? \"Resource\":\"*\"
? }]
}
"\
Then attach the role policy:
aws iam attach-role-policy --policy-arn arn:aws:iam::aws:policy/CloudWatchFullAccess --role-name CruddurTaskRole
Step 4-Create the backend flask task definition and service
task definition
? "family": "backend-flask",
? "executionRoleArn": "arn:aws:iam::023175542133:role/CruddurServiceExecutionRole",
? "taskRoleArn": "arn:aws:iam::023175542133:role/CruddurTaskRole",
? "networkMode": "awsvpc",
? "cpu": "256",
? "memory": "512",
? "requiresCompatibilities": [
? ? "FARGATE"
? ],
? "containerDefinitions": [
? ? {
? ? ? "name": "backend-flask",
? ? ? "image": "023175542133.dkr.ecr.us-east-1.amazonaws.com/backend-flask",
? ? ? "essential": true,
? ? ? "healthCheck": {
? ? ? ? "command": [
? ? ? ? ? "CMD-SHELL",
? ? ? ? ? "python /backend-flask/bin/flask/health-check"
? ? ? ? ],
? ? ? ? "interval": 30,
? ? ? ? "timeout": 5,
? ? ? ? "retries": 3,
? ? ? ? "startPeriod": 60
? ? ? },
? ? ? "portMappings": [
? ? ? ? {
? ? ? ? ? "name": "backend-flask",
? ? ? ? ? "containerPort": 4567,
? ? ? ? ? "protocol": "tcp",
? ? ? ? ? "appProtocol": "http"
? ? ? ? }
? ? ? ],
? ? ? "logConfiguration": {
? ? ? ? "logDriver": "awslogs",
? ? ? ? "options": {
? ? ? ? ? ? "awslogs-group": "cruddur",
? ? ? ? ? ? "awslogs-region": "us-east-1",
? ? ? ? ? ? "awslogs-stream-prefix": "backend-flask"
? ? ? ? }
? ? ? },
? ? ? "environment": [
? ? ? ? {"name": "AWS_COGNITO_USER_POOL_ID", "value": "us-east-1_JtCLJk9pn"},
? ? ? ? {"name": "AWS_COGNITO_USER_POOL_CLIENT_ID", "value": "oq199bt1i98d8fm471d8i32u4"},
? ? ? ? {"name": "FRONTEND_URL", "value": "*"},
? ? ? ? {"name": "BACKEND_URL", "value": "*"},
? ? ? ? {"name": "AWS_DEFAULT_REGION", "value": "us-east-1"}
? ? ? ],
? ? ? "secrets": [
? ? ? ? {"name": "AWS_ACCESS_KEY_ID" ? ?, "valueFrom": "arn:aws:ssm:us-east-1:023175542133:parameter/cruddur/backend-flask/AWS_ACCESS_KEY_ID"},
? ? ? ? {"name": "AWS_SECRET_ACCESS_KEY", "valueFrom": "arn:aws:ssm:us-east-1:023175542133:parameter/cruddur/backend-flask/AWS_SECRET_ACCESS_KEY"},
? ? ? ? {"name": "CONNECTION_URL" ? ? ? , "valueFrom": "arn:aws:ssm:us-east-1:023175542133:parameter/cruddur/backend-flask/CONNECTION_URL" }
? ? ? ]
? ? }
? ]
}{
service
? ? "cluster": "cruddur",
? ? "launchType": "FARGATE",
? ? "desiredCount": 1,
? ? "enableECSManagedTags": true,
? ? "enableExecuteCommand": true,
? ? "networkConfiguration": {
? ? ? "awsvpcConfiguration": {
? ? ? ? "assignPublicIp": "ENABLED",
? ? ? ? "securityGroups": [
? ? ? ? ? "sg-04ca5ebd69e0aec6f"
? ? ? ? ],
? ? ? ? "subnets": [
? ? ? ? ? "subnet-0462b87709683ccaa",
? ? ? ? ? "subnet-066a53dd88d557e05",
? ? ? ? ? "subnet-021a6adafb79249e3"
? ? ? ? ]
? ? ? }
? ? },
? ? "propagateTags": "SERVICE",
? ? "serviceName": "backend-flask",
? ? "taskDefinition": "backend-flask",
? ? "serviceConnectConfiguration": {
? ? ? "enabled": true,
? ? ? "namespace": "cruddur",
? ? ? "services": [
? ? ? ? {
? ? ? ? ? "portName": "backend-flask",
? ? ? ? ? "discoveryName": "backend-flask",
? ? ? ? ? "clientAliases": [{"port": 4567}]
? ? ? ? }
? ? ? ]
? ? }
? }{
Register Task Definition
aws ecs register-task-definition --cli-input-json file://aws/task-definitions/backend-flask.json
Create Services
aws ecs create-service --cluster cruddur --service-name backend-flask-service --cli-input-json file://aws/json/service-backend-flask.json
Step 5- Validation
Querying CloudWatch logs
aws logs describe-log-streams --log-group-name cruddur --log-stream-name-prefix backend-flask|grep logStreamName
aws logs describe-log-streams --log-group-name cruddur --log-stream backend-flask/backend-flask/cd05f3cfd3334b889d9a6aed43e31628
List tasks
aws ecs list-tasks --cluster cruddur
Connect to the container
This is the funny part.
First you need to install session manager plugin.
curl "https://s3.amazonaws.com/session-manager-downloads/plugin/latest/ubuntu_64bit/session-manager-plugin.deb" -o "session-manager-plugin.deb
sudo dpkg -i session-manager-plugin.deb"
After the installation, run the following commands for ubuntu.
aws ecs execute-command ?
--region $AWS_DEFAULT_REGION \
--cluster cruddur \
--task ab0324acd6b246009a394df074fc122d \
--container backend-flask \
--command "/bin/bash" \
--interactive\
I wanted to check the status of the /api/health-check endpoint.
.bin/flask/health-check.sh
I didn't create an application load balancer for now. After configuring the security group of my the RDS I could test the endpoints as shown in the images.
Clean-up
You can delete the service running the following command:
aws ecs delete-service --cluster cruddur --service backend-flask-service --force
Learn more about AWS Cloud Project Bootcamp
Andrew Brown is a AWS Community Hero, GCP Champion Innovator. You can follow him.
You can find the bootcamp's official playlist here. Unfortunately the registrations are closed but you can follow along the videos along with the bootcampers.
Summary
In this article I have demonstrated how to deploy a python flask app to AWS ECS fargate by:
See you in another project ??!