Context driven testing

The context-driven testing is a model based on the context of the project rather than going by books methodology testing or some fixed notion of best. It recommends testers to choose their testing techniques, test deliverables, test documentation and test objectives by looking into the details of a specific situation. Context-driven software testing is not a testing technique, it is a set of values about test methodology.

With this type of testing there is no single best practice that applies to all cases. Its value is contingent upon the amount of useful and timely information gathered during the testing process and its success depends on whether the software solution resolves the problem or situation it is intended to address.

Context-driven testers define their testing objectives, techniques, and deliverables by identifying the details of the specific situation such as desires of the stakeholders who commissioned the testing. The key strength of context-driven testing is project-appropriate application of skills and judgement rather than conventional methodologies.

Basic Principles of Context driven testing:

According to Cem Kaner, Brian Marick, James Bach and Bret Pettichord, the founders of Context driven school, the 7 basic principles in context driven testing are,

1. The value of any practice depends on its context.
2. There are good practices in context, but there are no best practices.
3. People, working together, are the most important part of any project’s context.
4. Projects unfold over time in ways that are often not predictable.
5. The product is a solution. If the problem isn’t solved, the product doesn’t work.
6. Good software testing is a challenging intellectual process.
7. Only through judgement and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products.

Context driven testing vs Standard testing

A traditional testing process in general follows a series of standard steps mentioned below to complete the testing,

• Requirement analysis
• Test Planning
• Test documentation – test scenarios, test case development
• Test environment setup
• Test execution
• Test reporting

These steps are decided based on the best practices, QA standards, industry guidelines etc. Now there is nothing wrong with this process. It will work fine and give the desired end result, as this is a time tested and standard approach. But if we raise a question whether this approach gives way for an optimal result, then the answer may be ‘doubtful’.

On the other hand context driven testing does not advocate following a set of same steps for every projects. As the name suggests, context driven testing gives more importance to the current context of a specific project rather than blindly following an established method/process. Because every project we deal with has different circumstances like,

• Documented or undocumented requirements
• Testing and development teams belong to different companies
• Tight schedules or relaxed schedule
• Testing tools used
• Availability/composition of testing team
• Type of the project etc.

Before deriving a test plan it is also important to ask questions like what, why, who, where, when and how to the stakeholders to know the project’s context.

Take an example of end to end testing of a healthcare application. The company wants to pursue context-driven testing to boost user satisfaction of its software product, minimize production defects and enhance the product by gaining a better understanding of the needs of users. Company has a short time frame for testing of the application.

In this case, “who” the application is intended to is the main question to be asked. Here the intended users are nurses and healthcare executives working on field. “How” and “where” these users use the application also becomes imperative. Nurses will mostly access it from hospitals and healthcare executives from remote locations. Asking these questions will help identifying the risks in different areas. Similarly, based on other information available for the project, context-driven test plan has to be derived. Since the schedule is tight for testing, it is also important to minimize the documentation and spend more time in testing the application.

So rather than following a test plan with specific lists of actions, testing teams are trusted to use their judgement and skill to determine how they need to test to meet the expectations of users. Successfully conducting context-driven testing involves identifying the intended market and evaluating the environment in which people are likely to use the software product. Context-driven testing is when you let these circumstances decide your test practices, techniques and even definitions rather than standard, industry-perceived ‘best practices’.

Values of Context-Driven Testing

Context driven testing is,

• Skeptical

Skepticism is not the rejection of belief, but a rejection of certainty. Skepticism helps in observing and analyzing more thoroughly, opening our minds to new and different ideas.

• Empiricist

In context driven testing, role of a human is considered vital. The experiences people involved bring to a project is of great value.

• Adaptable

Testing should be focused on adaptability, not creating as few tests as possible, but as many as possible, as challenging as possible. It is important to adapt to changing contexts and requirements.

• Diversified

It is important to consider how we can run more tests, more quickly, more powerful, while increasing our capacity to observe multiple quality criteria.

• Humanist

Testing must focus on human values. If quality is value to some person, then we need to understand people and their values. So it is a must to learn to observe people; customers, managers, programmers and project community.

• Heuristic

Context driven testing is heuristic as it does not follow any process blindly. It allows to discover and learn about the best approach to follow depending on the situation.

When to use Context Driven Testing?

Consider the following questions,

– Do you value more in individuals rather than their interactions over processes or tools?

– Do you value more in seeing working software over documentation?

– Do you value more in collaborating with your customers (including development) over negotiating contracts (and specs)?

– Do you value more in responding to change over following the plan?

If the answer to all the above questions is a “yes”, this is when context based testing approach can be followed.

It does not mean that this approach doesn’t believe in documentation. Documentation is important, but it values conversations more than that as it is clearer, faster and effective.

Context-driven testing will be used when one is trying to attain the objectives mentioned below:

• Clarify your objective.
• Supervise the major test planning challenges.
• Evaluate the product.
• Analyze any risk associated with the product.
• Set up logistics.
• Plan the test strategy.
• Share the idea.

When not to use Context driven testing

Context Driven Testing is not good for stable, predictable and repeatable projects; using it in such cases would be a waste of resources. There is no need of understanding a context in such cases and following context driven approach will only delay the project without adding any additional value to the end result.

Effective Context Driven Testing approaches

Context driven testing believes there are no best practices that is suitable in all conditions. But the following are few of the generally suggested approaches that help in effectiveness of context driven testing:

• Ask questions

It is very important to know the context of a project to get the maximum test coverage. So it is necessary for testers to ask questions to the stakeholders and developers, to understand the circumstances and factors involved to design a suitable test.

• Plan ahead

Planning helps to efficiently carry out the testing process. By creating a good test plan and sharing it with relevant stakeholders, helps them get an insight of what they can expect from it. It helps build rapport within the team and the company, invoking meaningful conversations.

• Adjust your plan

It is possible for software projects to not go entirely as planned. So one has to be ready to make adjustments as they arise. New features may be added, priorities may change and the plan should be adapted to these changes accordingly.

• Let stakeholders decide on the project completion

Stakeholders should be given the power and responsibility on deciding the timeline for project completion. This helps testers to focus on testing the software thoroughly and provide as much information back as possible.

• Avoid applying any practice blindly

This is the most important and obvious part on which the context driven testing is based. Best practices or a set of procedures will not work in every situation. Sometimes there is no time to make a detailed test plan ahead. In such cases, it does not make sense to use all the time to just plan and leave no time for testing. So it is essential to not just follow a practice blindly, but to do to what is effective for that particular scenario.

Summary

Context driven testing is an approach that advocates testing in a way that conforms to the context of the project. It declares that good testing is a matter of skill and not just procedure. Ultimately, context driven testing is about doing the best we can with what we get

Courtesy: Brainbox Testing CoE Team.