The Challenges of Agile Testing

Being a Quality Analyst/Tester do you feel unclear about your role in an Agile team? Are you doing less than what you might expect? Or are you being asked to do more in less time?

If you answered yes to any of the above questions, your experience is not unique. The impact of Agile on any team is huge. An Agile environment requires you to shift into a new role-a new way of doing things. As a QA tester, your role is extremely important to an Agile team and a misunderstanding of this role—or any role, in fact—in an Agile team could lead to failure or less than optimal results (or low ROI).    

I have been working with agile teams for a while now. In my experience, irrespective of the maturity and size of the team, as a tester, I have always come across following challenges:

DEFINING THE DONE CRITERIA

I think this is the most interesting and challenging part of AGILE methodology. This should be first and foremost process to be in place from Day1. Any deviation to this process can hamper the whole project deliverables drastically.

What is DONE Criteria ?

DONE criteria are high level check list items which needs to be satisfied in order to claim a story is DONE. DONE Criteria should have technical, process, functional, non-functional, configuration aspects of the software. DONE criteria forces teams to take holistic view on the stories rather then code completing the feature.

Some of the common aspects of done criteria

• All acceptance criteria met. (Functional)
• Performance not degraded. (Non Functional)
• Code coverage achieved and all acceptance tests automation. (Technical)
• All build passed and application deployed on UAT/Staging environment.   Environment/Configuration)
• Exploratory testing performed and QA signed off.
• Demoed to stake holders on proper environment. (Process)
• All known bugs closed and tested (Process)

By doing this teams will make sure that software produced is of great quality and all aspects of the applications are taken care. Doing this repeatedly will make sure testers are not doing catchup but they are part of the story and sign off process. Software produced in the end can be released and should be production ready.

SCRUMFALL

Many teams, a sprint often become a mini waterfall. It does not matter whether the sprint is a one-week, two-week or three-week sprint. The sprint can always become mini waterfall. In my early days as an agile tester, I used to wait for the stories to come my way and somehow all the stories would become available towards the end of the sprint. I struggled with it and tried to solve it in many different ways. In fact, with one of the team, we accepted it as a norm and assumed that first few days of the sprint, testers would work on the stories from last sprint. How naive, but hey that’s how we learn.

ENVIRONMENTAL COMPLEXITY

In many agile teams, continuous integration and deployment is a given. However, continuous deployment also means that environment is always moving. As a tester, it is important to be able to test in a stable environment whenever it is needed. However, sometimes it becomes difficult especially when we need to promote code to staging and production. We want to be confident that we are promoting the version we have tested. I have seen many solutions – from elegant build pipelines in Jenkins to stop-committing to master for a while policy. Solving this problem always becomes a problem because every application is different and environment often have many stakeholders (testers, developers, ops and so on).

FOCUS LOSS

As a tester in an agile team, it often becomes difficult to focus on testing. There are many things which require a tester’s time. Some of the things we need to work on are: prepare stories for the future sprint with business analysts, discuss stories of the current sprint with developers and BAs, keep an eye on broken build on CI, work with product owner for the demo, help developers with test data, environment set-up or unit testing, review unit, integration test, take care of non-functional testing when needed, test automation, support prod release, test in different environments and so on. This pace can often leave very little room for actual testing.

AUTOMATION OWNERSHIP

Test automation is integral part of agile teams. I have seen many successful and unsuccessful test automation efforts. In my experience, the biggest factor in the success or failure of automation effort is the ownership. If test automation is owned by the team, it has better chances of providing value. It’s difficult to promote this culture and it takes time. Often, testers in the team would want to treat test automation as their baby and would be reluctant to relinquish control and developers might shy away from taking additional responsibility. So it’s a difficult task to get everyone on board, but when the team owns test automation, it provides fastest feedback and adds maximum value.

SOLUTION TO THE CHALLANGES 

As a tester working in agile team, it’s important to ensure that project deliverables must be well defined, sprints don’t become mini-waterfall, there is always time for focused testing sessions and exploration remains integral part of testing. In an agile team, it helps if testers are not afraid of tools and technology. Agile teams are often tool-rich and testers need to get involved in every stage – from checking code quality, code coverage, functional and non-functional testing or may it be pair programming with developers to write negative test cases. This often makes the job of a tester a bit more demanding and exciting at the same time.

It is also important to think about and deal with the problems related to environment early, make team responsible for test automation and most importantly – keep on learning continuously and ask yourself – How am I adding value in the team?

Also feel free to share your feedback for:

• https://www.dhirubhai.net/pulse/moving-towards-agile-from-waterfall-prashant-kumar