Performance Test Execution Best Practices

It's always important to follow the best practices and when it is about the performance testing there is no room for error. Because during the performance test execution you have to gather a lot of information like the right load test environment, with right data combination, correct user load distribution, testing performance critical workflow, then, it should formulate correctly before running the test, and then, gathering a lot of raw data to analyze the performance bottlenecks. Which makes it important to follow the best practices. Meaning as a performance engineer, you have to make sure that you do all of it right so that you don't end up in running a different test with the same objective or running another round of performance test when you don't have any window of opportunity to do so.

The solution to make the execution smooth is simple, prepare a "checklist". You just need a checklist of things you have to do. It is more about to organize yourself and you can always automate the tasks. But, it is important to know what has to be done. So, to make it easy, I have divided the checklist into three phases:

• Pre-requisites or Preparation Activites Checklist
• Performance Test Execution Checklist
• Post-performance Test Checklist

These checklists are relevant to the performance lead or performance architect or performance engineer, a person who does the performance planning, execution, and communication. Also, we will not talk about workflow identification and load distribution or script creation in this post. It's assumed you have done all of that. Keeping technical tools and analysis skills off this post instead, we will focus on the smooth running of the performance test execution. So you will see the checklist that you should follow throughout Performance Test Execution

Pre-requisites or Preparation Activites Checklist

• Check the application URL that you are going to test: It will be too funny if you somehow manage to get the wrong URL tested, but can happen with anyone. So, be cautious!
• Check the test data used: It may take a while to verify the test data used for the testing purpose. But I have seen many people because of ignorance used wrong data because of which they don't get the desired results and regret later
• Check the required access to monitoring tools and active: Nothing better to have monitoring tools in place. If you have great, but check those are active and monitoring before executing the test
• Correct load distribution: It is important to double check if you have distributed the load properly. This is basic and important as you don't want to stress the workload or API that doesn't need to and don't want to leave the workload or API that is actually be utilized in the production.
• Ensure you have right people on board: Performance test involves monitoring and capturing different stats of different parts of the system and if that is the case then ensure you have the right team on board for quick measures
• Check the naming convention: Make sure the naming convention of the log files, results files, etc has some naming convention. If you run a lot of tests then you may get confused while analyzing
• Notify the stakeholders: Notify the test is about to start with a high-level summary is always appreciable

Performance Test Execution Checklist

There is nothing much here, because if you are into the test then there is nothing you can do. If anything goes wrong, you majorly have two choices either continue with the test or stop it make adjustments and rerun. But, I would suggest keeping a note of a few things while running a test.

• Check you are getting the correct hits: Ensure to check the results after 10-15 minutes in the test. If results look suspicious, check for the reason and if there are no anomalies in the performance then look or the test setup, basically ensure if you have everything in the place mentioned in the "Pre-requisites or Preparation Activites Checklist", if not, adjust.
• Monitor the results: Some think is it not important to monitor the test results while the test is running, that is not true! You should keep an eye to check how the test is performed. If the test is of like 8-10 hours duration then it is a different case, but to look at the results after a while is a good practice it gives confidence and you can observe things that otherwise get missed, for example, suppose a point at which the response time did not increase but the CPU shoots up, the chances are you likely to ignore.

Post-performance Test Checklist

• Notify the stakeholders: Notify the test is complete with a high-level summary is always appreciable
• Gather the result: This is the most emphasizing point. Make sure you gather all the data points you need. There are cases where the monitored data stored for a short duration of time or you are dependent on others to pass on the monitoring data to you like, in some cases, you may get the server stats from a operations team member. So, it's better to get access to the data as early as possible.
• Estimated hits to be achieved vs the actual hits achieved: Make sure you match the math you did, that is, ensure the target is achieved
• Find the root cause: If the test is merely to run to check the application can handle X amount of load then you can skip the root cause analysis but if the objective is to find the bottlenecks so that it can be fixed then jump and deep dive into the most interesting part of root cause analysis.

The above points are general from my checklist. Feel free to add if you think there are other points from your checklist.