Automation in Agile

There is no doubt that automation testing is primarily considered feasible if you are working on a Product. Moreover, the core feature set of that product should be functionally ‘Stable’. The areas which qualify for automated testing should not only be working properly but also least prone to any changes in near future. Otherwise, the cost of managing the automation scripts (and keeping those up to date) may seriously undermine the benefits promised by automated testing approach. 

Considering the above points, it seems that automation testing is merely applicable in ‘Agile’. However, this is not right. Automation testing can be applied on Projects as well as in ‘Agile’ model, with some preconditions fulfilled. Consider you are working on a project where the model is SCRUM, and project is to be delivered in 10 Sprints (normally a medium level project takes this long) where each sprint is of 2 weeks duration. To me, automation testing can be applied in this project as it spans over 5 months. 

Normally, first 3-4 sprints of such project will be quite hectic and there will be a lot of change requests from client after each build delivered. Consider the automation option completely off the table during this time period and focus on getting the core architecture stabilize first. Once the architecture of application is finalized and approved, then you can think about automation testing. During sprint 4-5 you will be having good knowledge of basic feature set of the application and many of those features will be functioning properly as well as will not be prone to changes (or major changes at least) in near future. This fulfills the ‘Preconditions’ I mentioned earlier. 

At this stage, in the ‘Gantt Chart’ of project, you are standing at 1.5-2 month passed and 3.5-3 months remaining. Now, it is the real test of your shrewdness, as you have to decide what scenarios should be automated. Here are few tips/suggestion that can be helpful: 

1.      Allocate a dedicated resource for automation, with sound expertise

2.      Review all the user stories of sprint 5-6 to 10 and get good idea of the ‘Big Picture’

3.      Consult other stake holders of the project and remove all major ‘gray areas’ for you in the project

4.      Perform impact analysis of future stories and try to thoroughly evaluate which user stories (already implemented or to be implemented both) are prone to changes

5.      Review the user stories implemented and evaluate what features are to be ‘Regressed’ in all the upcoming sprints

6.      Review the project and list all those features which will not be impacted by any future sprint 

Execution of point ‘3’, ‘4’ and ‘5’ will yield you the list of user stories/features which qualify for automation. So, let’s automate those all ….. right? My answer is ‘Not at all’. We are working in agile, and we have to invest our time in automation activity, in the most optimal way. So don’t go for automation of scenarios blindly, even though they qualify for it. Now we will further narrow down our list by considering few more things: 

1.      For which scenarios, my automation scripts will not require any maintenance effort throughout the rest of this project?

2.      For which scenarios, my automation scripts will require minor changes (e.g. due to upcoming features, my scripts will work but I will have to bind other data sets with it and will have to tune it slightly for some other environment etc.)

3.      For which scenarios, the required automation effort is minimal while it will save great amount of time for regression testing? (and will require no/least maintenance effort of course)

4.      For which scenarios, though the required automation effort is a bit high but it will save great amount of time for regression testing in all remaining sprints? (and will require no/least maintenance effort of course) 

I will automate those scenarios, which fulfill above 4 criteria completely. I will also repeat the same approach for remaining sprints of the project to identify and automate further scenarios/features. Both lists of suggestions can be extended depending upon other different factors of the project. However, all that I have presented so far can be considered as the Baseline for devising YOUR approach. My assumption was that project is of 10 sprints and its core architecture was finalized till 4th sprint at most, and I am the dedicated resource for performing the automated testing. If circumstances are different in your project, you may need to tailor the things accordingly. 

The Strategy:

There is one more important thing to consider. As soon as the architecture of application is finalized, The automation resource will start scripting the qualifying scenario from all previous sprints, while the rest of QA/Testing resources will work on manual testing of the user stories/scenarios of current sprint. Hence in each subsequent sprints, the automation resource will be 1 sprint behind rest of the QA team, as during a sprint it is the least possible thing to get the qualifying scenarios stable as well automate those. Still this approach will save the regression testing time in each next sprint more than before.

Conclusion:

In Agile, automation testing is not completely out of question. It can be applied but only for the scenarios selected in extremely wise way. Time invested in automation activity must return you the best possible output. Therefore, automate only those parts of the project which give you the best benefits of automation. Happy automation testing in Agile :). 

Regards,

Enjoy Knowledge Sharing.