Part 5 - Hands-On Test Automation Project with Java + Cucumber + Selenium + Spring + Docker

This article is a continuation of Part 4 - Reporting, Screenshots and Logging. The implementation of Part 4 can be found in my demo-spring-selenium project tagged as v4.0. You can download this release to use it as a starting point for this part of the article.

Part 5 - Parallel Testing and Docker

In many projects, we may end up with so many test scenarios that a full execution of test scenarios takes a long time. Sequential testing is time-consuming, and the faster we can get results the better. Reducing the execution time can be done by configuring parallel test execution. Another advantage of configuring test projects to execute in parallel is that if you can run test scenarios faster you will be able to add more coverage. 

Imagine the following situations: in your current project, your tests execute in 30 minutes. By parallelizing in 2 threads you may be able to execute in 15 minutes. Or if you execute them in 6 threads, you will be able to get results in about 5 minutes. Another possibility! Let's say you are testing just a single browser in 30 minutes. Then you start parallelizing in 2 threads, and each thread will execute a different browser. Or you can also run in 6 threads and each thread executes in 6 different browser versions or configurations. You will keep the same time to execute, but now you have more coverage.

Thinking in parallel execution since the beginning of your test project is a good practice.

Cucumber can be executed in parallel using TestNG by setting the DataProvider parallel option to true. In TestNG, the scenarios and rows in a scenario outline are executed in multiple threads. In our project, this can be easily configured in our CucumberRunner.java class by adding the following additional lines.

@CucumberOptions(
    glue = "demo/spring/selenium/stepdefinitions",
    features = "src/test/resources/features",
    plugin = {
        "pretty",
        "html:build/test-results/html-report.html",
        "json:build/test-results/json-report.json"
    })
public class CucumberRunner extends AbstractTestNGCucumberTests {

  @Override
  @DataProvider(parallel = true)
  public Object[][] scenarios() {
    return super.scenarios();
  }

}

The default thread count of the DataProvider in parallel mode is 10. To change this the dataproviderthreadcount property needs to be changed. The following command should be used to execute the tests with a different thread count. For testing purposes, execute the following commands. Using the first command you will see the scenarios being executed sequentially because we defined the thread as one (just as we have been executing until now). The second command line will execute the scenarios in parallel (two browsers will be opened at the same time).

./gradlew test -Ddataproviderthreadcount="1"


./gradlew test

Infrastructure is another something we should think about when we are setting up our automation project. In the beginning, it may be very simple to run the tests locally, but it is not viable in the long term. In this topic, we will use Selenium-Grid + Docker to complete our automation project structure.

Selenium-Grid allows you to run your tests on different machines against different browsers in parallel. That is, running multiple tests at the same time against different machines, different browsers, and operating systems. Essentially, Selenium-Grid supports distributed test execution (https://www.seleniumhq.org/docs/07_selenium_grid.jsp).

Docker is "an open platform for developers and sysadmins to build, ship, and run distributed applications, whether on laptops, data center VMs, or the cloud." (https://www.docker.com/). In other words, Docker can help us run our automation tests in a distributed way.

SeleniumHQ provides public docker images that we can use to run our tests (https://hub.docker.com/r/selenium). The images available are:

• Standalone – Images that create a standalone Selenium server. You’ll only be able to run one of these at a time on your local machine.
• Hub – Image that creates a central Selenium-Grid.
• Node – Images that are used in conjunction with the “Hub” image to create a Selenium-Grid. You can start multiple node containers that connect to your Hub image.

We will need to change our current project to be able to achieve the goals. The first step is to define the infrastructure that will support our test execution. Let's say we need to run our test on Firefox and Chrome. We will need an infrastructure that will create a docker container with the official Selenium Firefox image (selenium/node-firefox), a container with the official Selenium Chrome image (selenium/node-chrome), and connect those containers to a Selenium-Grid image (selenium/hub) container. It will look like this:

To accomplish this we can use Docker Compose which is a tool for defining and running multi-container Docker applications. With Compose, we use a YAML file to configure our application’s services. Then, with a single command, we create and start all the services from our configuration. It means we will be able to deploy and scale our complete infrastructure with one command.

For our project, we need to create a docker-compose.yml file on the root of our project. The file below is what will go inside of our file. 

version: "3"
services:
  selenium-hub:
    image: selenium/hub
    container_name: selenium-hub
    ports:
      - "4444:4444"
  chrome:
    image: selenium/node-chrome
    volumes:
      - /dev/shm:/dev/shm
    depends_on:
      - selenium-hub
    environment:
      - HUB_HOST=selenium-hub
      - HUB_PORT=4444
  firefox:
    image: selenium/node-firefox
    volumes:
      - /dev/shm:/dev/shm
    depends_on:
      - selenium-hub
    environment:
      - HUB_HOST=selenium-hub
      - HUB_PORT=4444

We can observe in this file that we have 3 services:

• selenium-hub: it uses the official Selenium Hub image.
• chrome: it uses the official Selenium Chrome Node and connects to the Selenium Hub.
• firefox: it uses the official Selenium Firefox Node and connects to the Selenium Hub.

 To test our docker-compose file open your terminal and do the following:

1. Access the folder that contains our docker-compose.yml file.
2. Execute docker-compose up -d
3. Access the Selenium Grid console: https://localhost:4444/grid/console
4. You should be able to see one instance of chrome and firefox available.
5. Execute docker-compose down (it will shut down the containers)

Let's scale more firefox and chrome browsers:

1. Access the folder that contains our docker-compose.yml file.
2. Execute docker-compose up -d --scale firefox=2 --scale chrome=2
3. Access the Selenium Grid console: https://localhost:4444/grid/console
4. You should be able to see two instances of firefox and two chrome available.
5. Execute docker-compose down (it will shut down the containers)

Once we have our docker-compose file with the infrastructure created we need to configure our project to bring the containers up and down before and after the tests respectively. We configure this directly on the build.gradle file using a docker-compose plugin for Gradle script.

Inside the buildscript in your build.gradle file add the dependency:

classpath "com.avast.gradle:gradle-docker-compose-plugin:0.13.0"

Also, apply the docker-compose plugin (place it right after "plugins").

apply plugin: 'docker-compose'

And then at the end of the build.gradle file add the following:

dockerCompose.isRequiredBy test

dockerCompose {
   useComposeFiles = ['docker-compose.yml']
   scale = [firefox: 2, chrome: 2]
}

We need to make a few adjustments to our project to initiate the browser instance pointing to our docker containers. We also need to make adjustments in the project so we can also execute the test scenarios in a chrome browser.

Let's start by changing the WebDriverManager.java class in a way that will allow us to instantiate the WebDriver using a RemoteWebDriver from docker containers. Create a method getRemoteWebDriver(browser) that will instantiate the remote driver accordingly to the browser we want to instantiate, passing the browser name as a parameter. 

In the webDriverFactory() method we just need to change it to call the new method getRemoteWebDriver(browser). The question that may arise now is: How will I inform which browser I want to execute the test scenario? Remember that is not a good practice to have configurations that we need to change constantly fixed in our code. We can add this information to our properties file. Add the property browser in the application.yml file, and create the new attribute inside DemoSpringSeleniumProperties.java class.

application.yml

browser: firefox

DemoSpringSeleniumProperties.java

  @Value("${browser}")
  private String browser;

Once we have the browser configuration ready we just need to instantiate the properties class with @Autowired inside our WebDriverManager and use the new attribute to pass the browser we want to execute our tests. The WebDriverManager.java class will look like the following:

@Component
public class WebDriverManager {

  private static final String DOCKER_HOST = "https://127.0.0.1";
  private static final String SELENIUM_PORT = "4444";
  private static final String CHROME = "chrome";

  @Autowired private DemoSpringSeleniumProperties properties;

  @Bean
  @Scope("cucumber-glue")
  public WebDriver webDriverFactory() throws IOException {
    return getRemoteWebDriver(properties.getBrowser());
  }

  private WebDriver getRemoteWebDriver(String browser) throws IOException {
    String remote = String.format("%s:%s/wd/hub", DOCKER_HOST, SELENIUM_PORT);
    if (browser.equalsIgnoreCase(CHROME)) {
      return new RemoteWebDriver(new URL(remote), new ChromeOptions());
    }
    return new RemoteWebDriver(new URL(remote), new FirefoxOptions());
  }
}

As you can observe, this method uses some constants defined at the beginning of the class. Constants are useful for defining values giving meaningful names to values that will never change. The CHROME constant is just to be used in the conditional to check which browser we should instantiate. The DOCKER_HOST holds the value to where we are running our docker (localhost), and SELENIUM_PORT the port we defined for Selenium in the docker-compose.yml (4444). 

An additional tip here is that the same way we added the property browser to the properties file, the docker host and selenium port can also be there in case you need the flexibility to execute the test scenarios in different hosts and ports.

The way we set up the project now removed completely the ability to run locally. However, running it locally is needed sometimes, especially when you are developing new tests. For example, if you try to execute the scenario through the IDEA instead of the command line it will fail because we automated the creation and removal of containers only through Gradle.

We can do a small change and have the ability to execute either local or remote. Let's add a context property in the application.yml file. If we pass "local" it instantiates a local Firefox, for example, and if we pass "remote" it means it will instantiate the RemoteWebDriver. Let’s create this property with a default value equal to remote. Make sure you also create the new attribute inside DemoSpringSeleniumProperties.java class.

application.yml

context: remote

DemoSpringSeleniumProperties.java

  @Value("${context}")
  private String context;

Once we have the context configuration ready we just need to a conditional statement in the webDriverFactory method inside our WebDriverManager. The WebDriverManager.java class will look like the following:

@Component
public class WebDriverManager {

  private static final String DOCKER_HOST = "https://127.0.0.1";
  private static final String SELENIUM_PORT = "4444";
  private static final String CHROME = "chrome";
  private static final String CONTEXT = "local";

  @Autowired
  private DemoSpringSeleniumProperties properties;

  @Bean
  @Scope("cucumber-glue")
  public WebDriver webDriverFactory() throws IOException {
    return properties.getContext().equalsIgnoreCase(CONTEXT) ? new FirefoxDriver()
        : getRemoteWebDriver(properties.getBrowser());
  }

  private WebDriver getRemoteWebDriver(String browser) throws IOException {
    String remote = String.format("%s:%s/wd/hub", DOCKER_HOST, SELENIUM_PORT);
    if (browser.equalsIgnoreCase(CHROME)) {
      return new RemoteWebDriver(new URL(remote), new ChromeOptions());
    }
    return new RemoteWebDriver(new URL(remote), new FirefoxOptions());
  }

}

Now we are good to execute by running a scenario through the feature file, or through CucumberRunner, or by command line.

To summarize, here is a list of possible ways of execution and outcomes using this test automation project (do not forget to also add this information in the README.md file):

1. Spring Profile: We can run either default or test. Nothing to be done when you want to execute the default profile, but for the test profile the environment variable SPRING_PROFILES_ACTIVE should be set as "test".
2. Context: We can either run local (local Firefox) or remote (in Docker containers). The default value is "remote", but for "local" the context property should be set as "local". In the command line, we pass the following argument: -Dcontext=local.
3. Browser: If we run remotely we have the option of running in a Firefox or Chrome browser. The default value is "firefox", but for "chrome" the browser property should be set as "chrome". In the command line, we pass the following argument: -Dbrowser=chrome.
4. Parallel: It is configured to run in parallel by default. The default threads count for parallel executions is 10. We can either change the thread count to 1 and execute them sequentially or even increase the default number if necessary. In the command line, we pass the following argument with the thread count: -Ddataproviderthreadcount="1".
5. Tags: All the scenarios are executed unless we specify the tag group that we want to execute. In the command line, we pass the following argument along with the tag(s): -Dcucumber.filter.tags="@smoke".

We are done with our automation project. In this article, we covered many points that should be taken into consideration when building automation projects. If you have practiced this article at this point you now have a structure that you can start creating your test scenarios, and it is robust enough that supports basic to complex scenarios with minor architecture adjustments, it has multiple ways of running the scenarios, and provides a good level of reporting and logging.

Remember to commit your changes to Git:

git add .
git commit -m "Parallel Testing and Docker"
git push

The implementation of this part can be found in my demo-spring-selenium project tagged as v5.0. You can download this release to check how it looks.

Thanks for reading! I hope you found it helpful.

If you enjoyed it, I appreciate your help in spreading it by sharing it with a friend.