Test Plan vs Test Strategy: What's The Difference?
Test Evolve
Agile Test Automation...Driving Quality Retail and Ecommerce Omnichannel delivery
In software testing, two key documents can play a crucial role in shaping the testing approach for a company and its projects - the organisational test strategy and the project level test plan. Understandably, there is often confusion both in and out of the test team about each document’s purpose and exactly what should be contained in either one. Despite their similarities, it is essential to note that they have distinctive objectives and varying areas of emphasis.?
In this blog post, we will explain the difference between a test plan and a test strategy, their importance, and how they complement each other.
What is an Organisational Test Strategy and why do you need one?
This is a high level, concise but technical testing guide that provides clarity and structure on how testing should be planned and executed within a company.
The Organisational Test Strategy exists at a corporate level, providing generic testing protocol for projects within its scope. It is not designed to be specific to any given project.
For a smaller company, the test strategy may cover all testing activities that may arise throughout delivery. A global organisation spanning numerous industry verticals may see fit to produce a number of Test Strategies as warranted, due to the differing nature of the respective development streams or the criticality of the products and software that each delivers. Likewise, differing areas of the business may author separate test strategies if one follows a waterfall model and another employs agile delivery methods.
Regardless, a strategy should contain practice and procedures for any type of anticipated testing sub-processes that will commonly exist at the underlying project level. For example, whilst not project specific, it can make test phase strategy statements about unit, integration, acceptance, accessibility, performance and automated testing amongst others.
What are the Advantages of a Test Strategy
Having a good test strategy that everyone follows shows that you care about making your products work well and that the big bosses agree on how to do it.
All the projects you work on will have a clear goal for testing from the beginning. The people in charge and the ones who make the products will see that you use the same testing methods for different things and that they match the rules set by the test strategy.
If a new team is having trouble figuring out how to test their stuff, they can just look at the strategy and get some help.
Key Components and Steps For Drafting a Test Strategy
Follow the below instructions to create a robust test strategy.
Outlines an approach to risk management to be adopted within the testing activities.
Describes the organisation’s approach to selecting and prioritising test execution, in the form of prioritised test procedures. Test procedures consist of prioritised test cases, derived from prioritised feature sets via prioritised test conditions and coverage items.
Identifies the documents expected to be produced during testing for the test project as a whole. Outlines when these are produced and the relevant signoff process. This is tightly connected to the test process that is specified in the policy.
Outlines the organisation’s approach to automated testing. Identifies the testing tools to be used throughout testing.
Describes the configuration management to be performed for the work products from testing; describes how these work products are to be identified, traced, stored, and made available to stakeholders.
Outlines how incidents should be triaged throughout testing.
Identifies specific lower level test phase sub-processes to be undertaken within organisation testing falling within the scope of the strategy.
Outlines the rules to ascertain when testing should start and stop.
A test sub-process consists of the following processes:
Different entry and exit criteria may be defined for each of these individually, or for selected ones, or for the entire sub-process as a whole.
Outlines how the company decides that the testing for the test sub-process to be finished.
Identifies the test documentation, including reporting, used for testing activities in the test sub-process.
Describes when each document or report is prepared and the associated approval process. This is tightly connected to the test process specified in the policy.
Sets the level of autonomy of the testers. States how this testing group is technically, managerially, and financially independent.
Identifies specific test design techniques to be used during test design and implementation within the test sub-process.
Identifies the test environment for the test sub-process; may state where specific types of test should be performed, and identifies groups or organisations responsible for the test environment.
May identify the origin of test data, state where particular types of test data are located, and groups or organisations responsible for test data.
Outlines the metrics to be collated and managed throughout testing sub-processes.
Documents the retesting and regression testing strategy, activities and conditions for any lower level test phases.
After developing an organisation test strategy, review the following in order to create a project level test plan.
Test Plan: Scope, Structure, and Evolution
The Test Plan provides a test planning and test management document. Some projects may have a single test plan, while for larger projects multiple test plans may be produced. Individual test plans can be utilised across multiple projects (at the programme level), or to a single project (project test plan/master test plan), or to lower level test phases (system test plan, integration software test plan, performance test plan, accessibility test plan, unit software test plan). If more software test plans are created, a mapping tree may be produced to aid documenting relationships and the information contained in each.
The Test Plan describes the decisions made during the initial planning and evolves as re-planning is performed as part of the control activity.
Advantages of a Test Plan?
A test plan holds enormous importance in software testing and clarifies the testing objectives, scope, and deliverables, enabling the testing team to align their efforts with the goals of the project.?
Some benefits of using a test plan:
Key Components and Steps for Writing a Successful Project Test Plan
Testing a product without prior knowledge can be troublesome. Examine the product's requirements, design, primary features, and operation in great detail.
A successful test plan requires a valid and authentic test strategy. Create a thorough test strategy with an overview of the testing methodology, approach, and strategies.
Identifies the project(s) or the test sub-process(es) for which the plan is being written and other relevant contextual information.
Clearly state the aim and approach of the test plan. It involves verifying specific functionalities, validating system performance, and ensuring compatibility.?
Identifies the test item(s) for the testing covered by this plan including their version/revision or reference where this information can be found. Scope summarises the features of the test item(s) to be tested. It will also document any test exclusions and reasons why.
Outlines any assumptions and constraints for the testing documented within this plan. These may include regulatory standards, the requirements in the Test Policy and the Organisational Test Strategy, contractual requirements, project time and cost constraints, and availability of appropriately-skilled staff, tools and/or environments.
Documents the stakeholders and their interest in the testing and outlines how communication with stakeholders will be undertaken.
Highlights the risks reviewed by the testing within this plan. This should include any relevant risks that may be specified in the Organisational Test Strategy. Provides an exposure level for each risk based on its impact and probability. Provides recommendations to treat the risks. This section may reference where a separate risk register can be found.
Identifies all documents that are to be delivered from the testing activity or equivalent information to be recorded electronically, for example in databases or dedicated test tools.
EXAMPLE The following documents could be included:
Documents which test design techniques can be employed.
Specifies the requirements that must be met before testing can start, such as the availability of a particular build or the conclusion of a list of required development tasks. Describes the conditions under which the relevant test organisation considers test execution activities to be complete.
Describes the metrics for which values are to be collected during the test activities.
Specifies all relevant test data requirements for the project or test sub-process (as appropriate).
Includes environmental requirements that ensure testing operations may be carried out successfully.
Specifies the conditions under which retesting and regression testing will be performed. May include anticipated test-cycle estimates.
Specifies the criteria used to suspend and resume all or a portion of the testing activities in the Test Plan. Identifies who is responsible for suspending and resuming testing activities. Specifies the testing activities that may have to be repeated when testing is resumed.
Records any Test Plan content that deviates from the Organisational Test Strategy. Identifies the authorities responsible for approving deviations, where applicable.
Identifies all necessary testing activities based on the test process to be used. Any retesting should also be considered alongside any general dependencies.
Describes the staffing requirements for the testing covered by this plan. Documents an overview of the primary and secondary teams undertaking the testing roles with their responsibilities.
Identifies specific requirements for additional testing staff that are necessary for the test project or test sub-process. Specifies test training needs by skill level and identifies training options for providing the necessary skills for the staff needed.
Documents testing phases and project checkpoints defined in the delivery schedule. Summarises the overall schedule of the testing activities, identifying where activity results feed back to the development, organisational, and supporting processes. Specifies the schedule for each testing phase based on estimates, resources and dependencies.
Bottom line
The test strategy explains the strategic approach and principles driving the testing activities inside for an organisation. A test plan offers the general structure and specifics of the testing effort for a particular delivery or phase of testing within a project. The test plan and strategy are both critical components of solid software test planning. They serve different roles but play a valuable part in ensuring a successful testing procedure.
Webinar - Designing your Agile Test Automation Strategy
Are you looking for ways to design your agile test automation strategy? Do you want to learn how to decide what and what not to automate, define critical test packs with risk scoring, and choose the best test frameworks, test data, test environments and test team for your project? If yes, then Watch this webinar on Designing your Agile Test Automation Strategy!