Test Automation - Optimizing Playwright Test Reports: Automating Allure Results Merging with GitHub Actions

Introduction

When working with Playwright for automated testing, especially in CI/CD environments, generating structured test reports is crucial for debugging and maintaining test quality. Allure Reports provide a detailed overview of test executions. However, when tests are executed in parallel using the GitHub Actions matrix strategy, merging multiple Allure results into a single report can be cumbersome.

A recently published GitHub Action, Allure Results Combiner and Publisher, simplifies this process, allowing seamless merging of test results across matrix jobs and automatic publishing to GitHub Pages. In this article, I’ll discuss how I leveraged this GitHub Action to optimize my Playwright test reporting workflow. I've shared the solution presented in this article in my Playwright Typescript example project.

The Challenge: Merging Allure Reports in CI/CD

When running Playwright tests in GitHub Actions using a matrix strategy, each shard generates separate Allure result files. These must be combined to create a unified HTML report. Previously, I manually implemented steps to:

• Download Allure results from multiple jobs.
• Merge them using bash scripts.
• Generate a final report.
• Deploy the report to GitHub Pages.

This approach, while functional, was cumbersome and error-prone.

What I Had to Do Before

Previously, my Nightly Regression Test Workflow looked like this:

While this worked, it required multiple manual steps and redundant scripting.

How the New Approach Simplified My Workflow

The solution code can be found here.

By integrating Allure Results Combiner and Publisher, I was able to:

• Eliminate the need for separate download, merge, and generate steps.
• Automate Allure report generation in one step.
• Automatically publish test history to GitHub Pages.
• Enhance debugging with linked commit and environment details.

This significantly reduced complexity and maintenance overhead and improved the reliability of test reporting.

Conclusion

The Allure Results Combiner and Publisher GitHub Action is a game-changer for Playwright test automation. It simplifies test reporting, improves debugging efficiency, and ensures a structured and accessible historical record of test executions.

If you're using Playwright with GitHub Actions, I highly recommend incorporating this action into your workflow to enhance your reporting capabilities.

Happy testing!