Building Blocks for In-sprint Automation

In the current Software Development era, continuous testing became more prevalent for early feedback on code quality. Hence, it’s important to automate test scripts in sprint as the product is being developed. Many agile teams mastered in In-Sprint automation and are able to integrate testing as part of build pipeline for instantaneous feedback. But there are teams who are still in process to migrate to In-Sprint automation. This journey might be challenging based on parameters like complexity of the product, technology stack, team’s readiness and upstream & downstream dependencies.

If you’re doing traditional automation and envision to migrate to In-sprint, integrated and continuous delivery of automation testing values, here are the building blocks for this migration - 

Identify Automation Scope

This step is the most critical piece for success of In Sprint automation. Below are the questions to outline the scope of automation –

• Should you consider sprint “N” or Sprint “N-1” test cases?
• Should you automate functional test cases or regression test cases or both?

If you are new to this concept, it’s recommended to go for “N-1” sprint regression test script automation. This will provide team to automate on complete and stable features.

Another recommendation for scope identification includes (1) Review Return on Investment (2) Analysis of automation feasibility and (3) Repeatability of scripts & functionality.

Also note that scope will include both new scripts and the scripts that need maintenance. Refer the below section [“Script Maintenance”] for more details.

 Commit Automation Effort in Sprint

Next step is to commit team’s bandwidth on the identified automation scope in the sprint. You should document scope and acceptance criteria in the product backlog in the form of an user story, then prioritize and commit the user story in sprint planning meeting.

Consider the following when you estimate your work [in Hrs./ Story Point / T-Shirt] –

• Does the script require framework level changes?
• Does it need specific Test Data or data set combination for execution
• Include automation scripting and code reviews
• Is there any downstream dependency? E.g. you might need to automate or virtualize downstream service calls.

Automation Scripting and Demo

This is the step where team builds automation scripts in sprint. If you have opted for “N-1” sprint automation, you can start automation scripting from Day 1 of the sprint. If you are doing “N” sprint automation, you will be collaborating with Development team and plan your automation tasks. You can also work to build stubs and structures so that you can quickly embed automation code as soon as feature is developed.

Code review is equally important and here are the types of reviews –

• Peer Review: Work with your peer for continuous feedback as you are building the scripts.
• Approver Review: If you’re using standard repo management, create pull request for your change and send to reviewer for approval. It’s recommended to have a code review checklist. This creates an awareness of the standard processes and reduces code review churns.
• Functional Review: This review is to make sure automation scripts have 100% functional coverage with respect to test scripts.

Once code review is complete and scripts successfully executed in lower environment, demo automation scripts to scrum teams.

 In-Sprint Automation Execution

In this phase, team realizes the value for In-Sprint automation. If you’re new to In-Sprint automation, enable nightly execution on lower environment for entire automation coverage for the application. You can also consider adding automation coverage as part of build and release pipelines for continuous feedback.

Script Maintenance

As you’re building automation scripts in parallel to the application being developed, it’s very likely that you’ll spend more effort in automation script maintenance. Continuous execution cadence in sprint will help to identify the scripts which have broken due to recent development work.

I recommend that you prioritize or provide equal priority to automation script maintenance alongside new script development. It’s also recommended to use similar process [as outlined in section - Commit Automation Effort in Sprint] to commit & account script maintenance effort in sprint.

Hope it provided you with some guidance to define a structure on how automation activities flow in sprint. I would love to hear back your feedback and your experience from In-Sprint automation.