Serverless load testing with Artillery

Serverless load testing with artillery

Performance testing, load testing, site reliability engineering, observability etc these are all the buzz words related to performance and monitoring an application in IT world. Many of us agree performance testing is required, the main purpose of performance testing is to identify and eliminate the performance bottlenecks in the software application. But if the application is hosted on a cloud provider having auto scaling, how do you test it?? For me, to test an application hosted on the cloud, I would like to be hands-on, identify bottlenecks, analyze logs, pinpoint issues and all of them should be available in one package instead of reaching out to another team for analyzing the problem. Artillery has a good, simple and quick solution. This article is very basic information about artillery and we have workshop planned to explain in-depth, details at the end of this page.

What is Artillery?

Artillery is a “modern, powerful & easy-to-use solution for load testing and functional testing”. It means it can be used to check the functionality of the application and also check the performance of the application by increasing the load to the same scenario.

The scenario is a setup of steps to simulate realistic load on the application. The syntax is pretty simple

• url — the request URL; it will be appended to the target
• json — a JSON object to be sent in request body
• body — arbitrary data to be sent in request body
• headers — a JSON object describing header key-value pairs
• cookie — a JSON object describing cookie key-value pairs
• capture — used to capture values from the response body of a request and store those in variables

The scenarios are customizable. You can define the target endpoint and also the probability of scenario being run. You can also request chain these endpoints to get the desired user flow and capture the expected response payload keys. This GitHub as good examples to build the scenarios.

What is Serverless?

Serverless by definition is an execution model where the cloud provider (AWS, Azure, or Google Cloud) is responsible for executing a piece of code by dynamically allocating the resource and only charging for the amount of resources used to run the code.

The Code is run inside a lambda like function and that can be triggered by a verity if events its sometimes referred to as?Functions as a Service.?Following are the FaaS offerings of the major cloud providers:

• AWS:?AWS Lambda
• Microsoft Azure:?Azure Functions
• Google Cloud:?Cloud Functions

These FaaS are designed to implement Event Driven Architecture. An event-driven architecture uses events to trigger and communicate between decoupled services and is common in modern applications built with microservices. Roughly speaking( for our quick understanding) we can divide AWS’s serverless building blocks into three categories:

• Compute - using lambdas
• DataStorate - using S3, dynamo db etc
• Application integration - step function, SNS, SQS, API Gateway etc

How to performance test on Serverless?

Web applications can experience high levels of traffic and peak load patterns and goal of testing is to ensure that the architecture and system design works for the amount of traffic expected.

When you are testing a serverless architecture using AWS, you can test the round trip from the AWS gateway. This will provide you a overall performance of the APIs that are being consumed by the UI layer but you can also isolate areas and the performance of each component. ApI testing provides the best approximation of the performance the users experience but testing isolated service would provide you with near accurate metrics.

Load testing?tools

There are several tools available for performance testing and one of the most popular is Artillery, this is an open-source tool for testing serverless APIs. You configure the number of requests per second and overall test duration, and it uses a headless Chromium browser to run its test flows. The maximum number of concurrent tests is constrained by your local computing resources and network in Artillery. You can use Serverless Artillery to achieve higher throughput which runs the Artillery package on Lambda functions and you scale up the tests significantly.

Here is an article published on AWS using Artillery to load test an application having serverless backend.

Artillery Plugins

Artillery has a host of plugins which can be used in conjunction with it to extend its functionality, for example:

artillery-plugin-cloudwatch — A plugin for?artillery.io?that records response data into AWS CloudWatch.

artillery-plugin-sns — A plugin for?artillery.io?that publishes response data to an AWS SNS topic.

artillery-plugin-dynamodb — A plugin for?artillery.io?that publishes stats to AWS DynamoDB.

Reporting and Monitoring

Artillery provides build in tests using html and also you can hook up reports to Cloudwatch and configure Cloudwatch dashboards to view your performance metrics.

I am co-host for Testaholics group and we are hosting a workshop (online) where we will provide repos and step-by-step instructions to get you set up-and-running powerful performance tests in minutes for a beginner. Additional information about the meetup group is provided here.

Thank you for reading !!!!