Why Keyword Driven Test Automation is not way forward for Agile

This post is the first article in a three article series where I discuss about approaching test automation and why conventional methods might not be the way forward, specifically in agile environments.

Keyword driven test automation frameworks are a great way to accelerate test automation, and to utilize testing resources with limited or no scripting knowledge to create automation scripts.

Although Keyword driven frameworks can help accelerate in certain situations, there are certain aspects of Keyword driven approach which makes it unusable or inappropriate for organizations following an agile development approach.

Popular approach to Keyword driven frameworks need to have a set of keywords defined either based on:

• the application functionality or
• the low level/action level of the automation engine

We will discuss each of the above approaches in separate posts, since there are basic differences in the issues these approaches present and handling them as part of future maintenance

Keyword driven frameworks/tools which need the keywords defined, which will later be used/called to create test scripts, require the application functionality clearly defined in terms of different flows it contains. Once all the flows of the application are identified, the Keywords are defined and later implemented. This approach to automation ends up investing too much time upfront identifying the application flows, and any change to the application flows would require not only updating the implementation of the Keywords themselves, but also all the test scripts that are impacted by the application changes.

This ends up being too much of maintenance activity with too little traceability to the actual user stories(and changes). Although the assumption is that Keyword driven frameworks can be utilized by non-technical resources in the team to create the tests(and to reduce the costs related technical resources), it might not be as cost effective.

Another issue with this approach is that the test teams still need to maintain two different assets(manual test cases and automation tests) since although the keywords(and test scripts) are readable, they are not readable enough most of the times, and leave too many gray areas for assumption to use them as manual tests.

Now the test teams have to trace changes to user stories(and application behavior) to update Manual test cases and automation test scripts, which as evident is too much of maintenance activity.

Since this is a Shift Right approach(instead of Shift Left), the automation is always behind the sprints(or development) not considering a change in application functionality. Once a major change in application functionality occurs, maintenance activity kicks in, taking lot of time to make fixes and have them working again(this means automation falls way behind the development)

I will in the next post, discuss about the Keyword driven test automation with tools/frameworks that provide a way to create test scripts with low level/action level steps(based on underlying automation engine) and what issues it presents in Agile environment.

I would love to hear from you, What in your experience, have been the disadvantages of using Keyword driven frameworks in an Agile environment?

Disclaimer: The opinions in the post are purely based on my personal experiences and do not represent that of my employer.