Cloud provides auto scaling features, but will your application auto scale??

While cloud technologies have solved traditional challenges like lengthy procurement times, high initial costs, data centre maintenance issues etc, there are new challenges which need to be addressed with these new cloud technologies.

Few of the new challenges with cloud are as mention below..

1.   Proper bench-marking tests should be conducted to find the best balance between price and performance. Over provisioning of cloud resources could be too expensive over time.

2.   Cloud provides auto scaling features, but will your app really auto scale? Is your application configured with proper auto scaling alarms? Have you configured auto scaling alarms based on CPU or Memory or requests per second? How do you decide on what resource auto scaling alarms should be based on?

3.   Do you have proper throttle in place to protect down-stream systems from crashing if the up-stream auto scales unexpectedly with spiky load?

4.   Even Serverless apps (AWS lambda for example) should not be deployed into prod without proper Performance Testing because, it could lead to flooding of down-stream systems if there are any issues with the proposed design. And, the account concurrency could be maxed out as every AWS account has a limit on the number of concurrent Lambda functions that can run.

Performance Testing tools can simulate the workload expected on our IT systems using virtual users so that we can measure our system performance and can tune the performance issues if there are any. We can also figure out the required optimal configurations for our application so that it will perform at its best.

Based on this simulated user load, we can also estimate the required IT resources to be provisioned like CPU, Memory, Network bandwidth required, Amount of storage required, No. of server instances to be provisioned etc. Performance Testing will help to avoid the risk of under provisioning IT resources and can save the costs that might arise due to over provisioning of IT resources.

Performance Testing will help to decide what auto scaling alarms are required for your application and on what resource (CPU / Mem).

Here is an example of sample auto scaling recommendations for Microservices deployed on AWS ECS derived as an outcome of "Performance Testing" engagement.

Assumption: We will be having one container in one ECS Task.

EC2 instance type to be used: T3 Small

• CPU Soft limit : 100 units per task
• CPU Hard limit : 500 units per task
• Memory Soft limit : 256 MiB per task
• Memory Hard limit : 512 MiB per task

Auto scaling limits for ECS Tasks - To be set at Service level

• Min tasks = 2 
• Max tasks = 4

Auto scaling limits for EC2 instances - To be set at ECS Cluster level

• Min EC2 = 2
• Max EC2 = 10

ECS Cluster level alarms:

Alarm 1 - Based on Memory

• Scale up - Setup auto scaling alarm to trigger when the Cluster Memory goes above 80%
• Scale down - Setup auto scaling alarm to trigger when the Cluster Memory goes below 32%

Alarm 2 - Based on CPU

• Scale up - Setup auto scaling alarm to trigger when the Cluster CPU goes above 70%
• Scale down - Setup auto scaling alarm to trigger when the Cluster CPU goes below 28%

Service level (Tasks level) alarms:

Alarm 1 – Based on CPU

• Scale up - Setup auto scaling alarm to trigger when the Service CPU goes above 95%
• Scale down - Setup auto scaling alarm to trigger when the Service CPU goes below 38%

Please Note: These recommendations are not generic so should not be considered for other micro services without proper validation using Performance Testing as CPU and Memory requirements vary from service to service.