Part 4 - Hands-On Test Automation Project with Cypress + Cucumber + Docker

This article is a continuation of Part 3 - Configuration. The implementation of Part 3 can be found in my demo-cypress project tagged as v3.0. You can download this release to use it as a starting point for this part of the article.

Part 4 - Reporting

It is very important to have a report on our test automation project. A good report allows us to evaluate the current status of the project and the overall quality of the product. It makes it easier to take corrective actions if it is necessary. The test report can also determine if the product is ready for release or not. The key thing on setting our report is to make sure that the report generated will give visibility to stakeholders (team, product owners, managers, etc.) of the tests that passed and/or failed, and if it failed for what reasons. 

Accordingly to Cypress documentation, Cypress is built on top of Mocha, which means any reporter built for Mocha can be used with Cypress. Here is a list of built-in Mocha reporters.

They have also added the two most common 3rd party reporters for Mocha. These are built into Cypress and we can use them without installing anything.

• TeamCity
• JUnit

Finally, Cypress supports creating our custom reporters or using any kind of 3rd party reporter.

It is always great when we have the option to view the results of our test scenarios in an HTML format. I always check on how to integrate HTML reports on all of the automation projects I build. The main reason is not only because it is nicer to view the results, but it is also helpful when we need to share the results with non-technical people like Business Analysts, Product Managers, etc.

Jenkins has a very nice Cucumber report plugin that generates reports in HTML format. I think all automation projects should be integrated with a Continuous Integration tool like Jenkins to run the tests and have the reports saved. To be able to integrate our project with this plugin what we need to know is that this plugin uses a report in JSON format as an input to generate a nicer HTML report version.

For this reason, we need to configure our project to generate the report in JSON format, so later on we will be able to integrate this report on Jenkins and have an HTML view of the results.

On the package.json file we should add the following code. We are just informing that we want a cucumber json report generated on the folder "cucumber_report".

"cypress-cucumber-preprocessor": {
  "cucumberJson": {
   "generate": true,
   "outputFolder": "cucumber_report",
   "filePrefix": "",
   "fileSuffix": ".cucumber"
  }
}

Remember to add the report folder "cucumber_report" to .gitignore file, so the results of local executions are not committed to the repository.

That is all we need to do. Try running the tests again and we will see that a .json file with the results will be generated on the cypress/cucumber_report folder.

We usually do not need to have HTML reports generated locally. But in case we want to generate an HTML report locally there are some plugins to do this as well. They also rely on the .json file format to generate the HTML report, so the steps above are still needed. There are two that I have used already:

• cucumber-html-reporter
• multiple-cucumber-html-reporter

The configuration of both are very similar and simple to do. We just need to install it into our project and create a .js script that will generate the report. The documentation of both has an example of the script.

As an example we will configure the cucumber-html-reporter plugin which at the time I am writing this article it seems to be the most downloaded. The first thing is to install the package through npm.

npm install cucumber-html-reporter

Create a report.js file on the root of the project with the code below.

// Prepare folder structure

const fs = require('fs-extra');

const dir = './cucumber_report/html';

 

if (fs.existsSync(dir)){

  fs.removeSync(dir);

}

fs.mkdirSync(dir);

 

// Report configuration

const reporter = require('cucumber-html-reporter');

const options = {

  theme: 'bootstrap',

  jsonDir: 'cucumber_report',

  output: 'cucumber_report/html/cucumber_report.html',

  reportSuiteAsScenarios: true,

  scenarioTimestamp: true,

  launchReport: true,

  metadata: {

  }

};

 

// Generate report

reporter.generate(options);

Then execute the command below to generate the HTML file:

node report.js

We are done with Part 4 of this article. :)

Remember to commit your changes to Git:

git add .
git commit -m "Adding Reporting"
git push

The implementation of this part can be found in my demo-cypress project tagged as v4.0. You can download this release to check how it looks and/or use it as a starting point for the Part 5 of this article. 

Thanks for reading! I hope you found it helpful.

If you enjoyed it, I appreciate your help in spreading it by sharing it with a friend.