k6_Integration with GitHub Actions_Grafana_Reporting_Publish Results

Grafana k6 + GitHub Actions

• Grafana k6_Introdcution :- k6 is an open source load testing tool to test the performance of APIs, microservices, and websites. Developers use k6 to test a system’s performance under a particular load to catch performance regressions or errors.
• GitHub Actions Introduction :- GitHub Actions is a tool that enables developers to create custom workflows for their software development lifecycle directly inside their GitHub repositories. Integrating k6 performance tests into a new or existing GitHub Actions pipeline is quick and easy, especially using the official marketplace app. Grafana k6 has two official GitHub actions available in the GitHub marketplace to make it easy to run performance tests in a GitHub action workflow. The first is setup-k6-action , which is used to configure k6 in the workflow pipeline, and the second is run-k6-action , which is used to execute the k6 tests.

Writing the first Performance Script :-

• An HTTP request against our system under test (SUT).
• A load configuration controlling the test duration and amount of virtual users.
• A performance goal, or service level objective (SLO), defined as a threshold -Pass/Fail criteria with thresholds in your k6 script. k6 evaluates them during the test execution and informs about the threshold results. If any of the thresholds in our test fails, k6 will return with a non-zero exit code, communicating to the CI tool that the step has failed.

Thresholds are a powerful feature providing a flexible API to define various types of Pass/Fail criteria in the same test run. For example:

The 99th percentile response time must be below 700 ms.

The 95th percentile response time must be below 400 ms.

No more than 1% failed requests.

The content of a response must be correct more than 95% of the time.

Setting up the GitHub Actions workflow

• To have GitHub Actions pick up and execute our load test, we need to create a workflow configuration and place it in .github/workflows. Once this file has been pushed to our repository, each commit to our repository will result in the workflow being run.

We can define the Git Branch Name - and the Pull Request - Branch name - for which actions which we wanted to trigger the workflow . We can also define Cron Expressions inside the on Schedule - You can use on.schedule to define a time schedule for your workflows. You can schedule a workflow to run at specific UTC times using POSIX cron syntax . Scheduled workflows run on the latest commit on the default or base branch. The shortest interval you can run scheduled workflows is once every 5 minutes.

This example triggers the workflow every day at 5:30 and 17:30 UTC.

GitHub Actions CRON Schedule

k6 has official GitHub actions to make it easy to execute your load tests. They are divided into two parts:

• Setup k6 action : This action allows you to install and configure k6 in the pipeline easily. You can use it to customize what version of k6 you want or optionally configure a browser to easily run k6 browser tests.

You can use the following options with this action to modify its behavior:

• k6-version: Specify the k6 version to use – for example, ‘0.49.0’. If not set, the latest k6 version will be used.
• browser: If set to true, a chromium browser is set up along with k6, allowing you to run k6 browser tests . By default, it is set to false.
• Run k6 action : This action allows you to execute k6 tests with ease. You can use it to execute multiple test scripts in parallel, run cloud test runs, automatically add a comment on a PR with URLs to test run results, and more.

The following action inputs can be used along with the action:

• path: Glob pattern to select one or multiple test scripts to run.
• cloud-run-locally: If true, the tests are executed locally and the results are uploaded to Grafana Cloud k6 , our fully managed performance testing platform powered by Grafana k6. By default, it is true.
• parallel: If true, and multiple tests are executed, all of them run in parallel.
• fail-fast: If true, the whole pipeline fails as soon as the first test fails.
• flags: Additional flags to be passed on to the k6 run command, e.g., --vus 10 --duration 20s
• cloud-comment-on-pr: If true, the workflow comments a link to the cloud test run on the pull request (if present). By default, it is true.
• only-verify-scripts: If true, only check if the test scripts are valid and skip the test execution.

Running cloud tests

There are two common execution modes to run k6 tests as part of the CI process.

• Locally on the CI server.
• In Grafana Cloud k6, from one or multiple geographic locations.

You can also run k6 tests locally on the CI server, and then push the results to Grafana Cloud k6.

Now, we will show how to run cloud tests using GitHub Actions. If you do not have an account with Grafana Cloud already, you can sign up for a free one today.

After that, get your account token and add it to your GitHub project’s Secrets page by going to Settings > Security > Secrets and Variables > Actions > Secrets.

We also need to specify the project ID where the test run should be stored. To do this, grab your project ID and store it as a GitHub action secret or define it directly in the workflow.

We will pass the token and project ID to the k6 action via environment variables.

Pass the k6_Cloud_Token which we will generate from Grafana Labs and after placing it inside the GitHub Actions -> Secrets - pass it inside the GitHub Actions workflow yml file :-

The only change in the workflow compared to running test locally and pushing results to the cloud is to set cloud-run-locally for the action to false. This will now execute the test on Grafana Cloud k6.

Attaching the full version of k6_GitHubActions -workflow yml file -

name: k6 Cloud Load Test

on:
  push:
    branches: [ main, master ]
  pull_request:
    branches: [ main, master ]
  schedule:
    # * is a special character in YAML so you have to quote this string
    - cron:  '30 5,17 * * *'  

jobs:
  run-cloud-test:
    runs-on: ubuntu-latest
    env:
      K6_CLOUD_TOKEN: ${{ secrets.K6_CLOUD_TOKEN }}
      K6_CLOUD_PROJECT_ID: ${{ secrets.K6_CLOUD_PROJECT_ID }}
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Setup K6
        uses: grafana/setup-k6-action@v1
      - name: Run local k6 test
        uses: grafana/run-k6-action@v1
        
        with:
          path: |
            ./specs_folder/k6_Cloud_Demo.js  
            ./specs_folder/Threshold_example_test.js 
          #flags: --vus 10 --duration 20s # optional: flags to pass to to each k6 test (default: none)  
          parallel: true # optional: run tests in parallel (default: false)
          fail-fast: false # optional: fail the step early if any test fails (default: true)
          cloud-run-locally: false
          cloud-comment-on-pr: true        

Grafana k6_GitHub_Actions_k6_Workflow_Definitions_GitHub_Actions_MarketPlace -

The following link contains all the steps to trigger the Git Hub Actions Workflow - and all the steps defined :-

Trigger the Workflow :-

• Once the commit is made to the branch name - defined in the workflow - the GitHub Actions workflow will get triggered and it will execute the test cases defined inside the with -tag .
• All the test cases will get executed inside the Actions tab -> and the Performance test results will be displayed.
• Grafana Cloud Execution will also take place - for the Project ID configured inside the Workflow.

Conclusion :-

• With the help of k6 - Integrations with GitHub_Actions - workflow we can easily trigger the Performance tests and execute the Regression Progression Test Scripts - to ensure there is no impact to any of the exisitng Perfromance test -whenever any new Progression test scripts is added or whenever any new commit is made to the branch.
• We have also seen how we can visualise the Performance tests with the Integraion of Grafana Labs - where we can get the visualisation represenation of the test cases execution.

Reference Links :-

Grafana k6_Actions - https://github.com/marketplace/actions/run-grafana-k6-tests

Grafana_k6_GItHub_Actions_Document - https://grafana.com/blog/2024/07/15/performance-testing-with-grafana-k6-and-github-actions/

GitHub Actions Syntax - https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions

Git Hub Repository - https://github.com/log2jiten24/k6_Performance_Test_Jiten

Git Hub_Secrets_Manage Document - https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions