JMeter Master-Slave Configurations using EC2 Instances

Introduction

Mimicking Real World Scenarios with JMeter

JMeter is an open-source performance testing tool, mainly used for testing web applications. It comes with both GUI and a CLI-based option to run tests. In performance & load testing, you’re basically trying to mimic a real world scenario of users entering a website. When a user visits a website, a portion of the resources are allocated towards accommodating that specific user. To mimic such scenarios, we make use of JMeter. When we run tests on JMeter, it makes use of “threads”, which are basically virtual users. In other words, a single user is represented by a single thread. With the help of these threads, you can mimic scenarios of thousands of users entering a website, and assess the performance of the website.

Scaling Threads to Avoid Overloading

In JMeter, whenever you run a script with threads, each thread consumes resources. That resource is allocated by the system that runs the JMeter script. At the start this is not a big concern, however, as you begin scaling your scripts and increase your threads, the system will start to get overload. To avoid this, we use the concept of Master-Slave Configuration. To keep things simple, just think of this configuration like a manager dividing work and handing it over to the employees. Let’s say the manager wants to run 300 threads, and has 3 employees. If the manager adopts the master-slave approach, he would give 100 threads to each employee to run. This way the load is split, and no one employee will be over-burdened. Now think of the employees like the systems running the scripts.

For this article, we will be using one master node (an EC2 instance) and one slave node (also, an EC2 instance). If you are able to execute all the steps mentioned in this article, increasing the number of slave nodes and applying the same concept shouldn’t be a problem for you.

Prerequisites

In this article, we will be making the assumption that the following have already been catered for:

1. JMeter is already installed on the Master Node and the Slave Node(s).
2. The JMeter version is the same across the Master and Slave Nodes.
3. The Java Version is the same across the Master and Slave Nodes.
4. The .jmx file to be executed is already present in the Master Node’s directory.
5. Firewall is configured in such a way that it does not block the communication between the Master and the Slave.
6. SSL will be disabled for RMI, in order to achieve simplicity in the configuration.
7. Both the EC2 instances are in the subnet and/or can connect to each other.

Finding the IPs of the Master and the Slave Node

Before we take any further steps, we should know the Public and Private IPs of the nodes involved in this configuration. If you already know the IPs, you may skip this part.

To check the Private and Public IP of your EC2 Instances, you will have to head over to the AWS Dashboard, go to the EC2 Page, and then click on your instance for more details. Then you will be greeted with a page that presents you both the public and private IP addresses of your node.

Pinging the Slave from Master Node

In order to ensure that our Master Node EC2 instance can reach the Slave Node EC2 instance, a good test is to try and ping from the Master Node to the Slave Node.

The command to ping is as simple as:

> ping [slave node’s public/private IP]

Note: When using EC2 instances in the same subnet, you can always ping the nodes using their public IP, however it is a good practice to use their private IPs.

If you don’t see the above screenshot or are not able to ping from the Master Node to the Slave Node, then it could be due to several reasons. Some of which could be, the EC2 Instances not being in the same subnet, NAT Gateway not being configured in the VPC, or Security Groups not allowing you to ping.

Setting up the Slave

Traditionally, to set up the slave node, we would first generate its RMI Key and then set it up for listening using the “jmeter-server” command. However, in this tutorial, we will be taking a far simpler approach and bypassing the need for an RMI Key. We would simply be running the following command:

> ./jmeter -s -Jserver.rmi.ssl.disable=true -Djava.rmi.server.hostname=[Slave’s Private IP] -Djava.rmi.server.useLocalHostname=false

The above command is different from the traditional “./jmeter-server” command, in a way that the “-s” flag allows us to configure the additional parameters that will help us to bypass the RMI Key. Meanwhile the “java.rmi.server.hostname” is there to help the nodes locate each other during the communication. Lastly, the “java.rmi.server.useLocalHostname” is set to false so it can use the parameters in “java.rmi.server.hostname”.

The above code disables the RMI SSL, which will make it not ask for a key and start the server anyways. Your slave node is successfully listening if you see a similar output to this:

Setting up the Master

Coming over to the Master Node, you will first need to specify the IP of the slave(s) that you will be using for the distributed testing. For this, we will first need to head over to the directory where JMeter is installed. Over there, you will notice a file by the name of “jmeter.properties”. We will need to edit this file. In our case we will simply do the following:

> nano jmeter.properties

After that you should see the something similar to the following screen:

We will scroll down all the way to “Remote hosts and RMI Configuration”. Under there we will find a variable by the name of “remote_hosts”, if it is commented, uncomment it and add the IP of your slave(s). It doesn’t really matter whether you are using Public or Private IP, but we recommend you use the Private one to remain synchronized. In case you are using more than one slaves, you will have to separate their IPs with a comma. Let us assume the IPs of our slave nodes are 1.1.1.1 and 2.2.2.2, then our variable should look something like “remote_hosts=1.1.1.1, 2.2.2.2”.

Since we are using a single slave node in this tutorials, so our configuration looks like this:

Then you can save the file and exit from the nano mode (in our case it was Ctrl+O then Ctrl+X).

Now that our jmeter-properties knows the IP of our Slave Node(s), we are good to execute the JMeter file.

Executing the JMeter File

Assuming that the JMeter file already exists in the Master Node, and that the Slave Node is successfully able to listen. We can now proceed to the execution. For that we will need to head to our Master Node, and run the following command:

> ./jmeter -n -t [Path to JMX File] -l [CSV File to store Summary] -r -Djava.rmi.server.hostname=[Public/Private IP of the Master Node] -Djava.rmi.server.useLocalHostname=false -Dserver.rmi.ssl.disable=true

In the above command, we are running JMeter in the Non-GUI Mode, using the “-n” flag. Then we specify our Path to the JMX File using the “-t” flag. We also specify the csv file to where our summary report will be stored, using the “-l” flag. The “-r” flag lets it know that it is a distributed test, hence it will use the parameters we provided in the jmeter properties file. The “-D” flags followed by the java parameters are there to help the nodes to locate each other during the communication, which we already covered above.

After your run the test, your Master Node’s screen should look something like this:

Moreover, on your Slave Node you can confirm as well that it has ran the tests. You should see something similar to this on your Slave Node:

Note: The JMeter file will not automatically distribute/divide the threads, in case of multiple slave nodes. Instead, it would run the same amount of threads in all your slave nodes. This means, if my JMeter File has a Thread Group that I have specified 10 threads for. That file will run 20 threads if I am using 2 slaves. This is because the same amount of threads will be run across the nodes, not automatically be divided. Do keep this in mind and not assume that JMeter will automatically split the threads during the Distributed Testing.

Conclusion

As your tests will start to scale, you will notice that a single machine might start running into problems while executing it, hence sooner or later you will need to implement distributed testing. The main goal in performance testing is to simulate users joining a specific application and collecting the data regarding the effects. As your application scales, you have to scale up the simulation as well and that is all possible thanks to distributed testing. Learning such types of architecture and distributed testing can open doors to cloud based testing, just like how we demonstrated in this tutorial. Personally, I believe that every person that is interested in performance testing should know how to implement this type of setup, so that it helps them with scalability and opens avenues to problem solving skills.

If you have any questions regarding this setup, feel free to reach out to me on my Linkedin and I will try to answer your queries to the best of my knowledge. My Linkedin: https://www.dhirubhai.net/in/ahmed-jawad-cs

Find this blog on our Medium, and many more. Happy reading!