Automation Testing Reporting Best Practices and Templates

Various documents and reports are made to be a part of Testing. Automation Testing Reporting can’t work without these. A few are the Test Strategy doc, Test Plan doc, Risk handling plan, Configuration Handling Plan etc. Out of these, a test summary report is made after the Testing is completed.

I have explained the entire reason behind?the ‘Test Summary Report’?and provided a sample Test Summary Report template with an accurate report for download.

Test Summary Report is an essential deliverable made at the end of any Testing Project or, most probably, after the Testing is over. The main motive of this document is to explain different details or activities regarding the stakeholders, senior management, clients etc.

Being a part of the everyday Status Reports, the expected testing results will be shared with the help of involved stakeholders each day. Whatever the case, the Test Summary Report will provide a comprehensive report of the Testing performed for the Project.

This artefact also needs to be an inseparable portion of the CMMI procedure.

What does the Test Summary Report Consist of?

A typical Test Report template has the information below, but the contents will keep changing based on every company’s format and practice. I have also given actual examples for proper understanding.

What are the 12 significant steps for writing an effective test summary report?

1st Step

Purpose of the Document

<A mini description regarding the objective behind preparing the doc.>

Like, This document properly talks about the different activities performed as part of the ABCD Transport System app Testing.

2nd Step- The overview of the App

<A short description of the App that is tested>

Just like, ‘ABCD Transport System’ is a website-based bus ticket booking app. Tickets for different buses can be quickly booked using online facilities. Actual time passenger info is received from a ‘Central Repository System’, which will be referred to before the booking gets confirmed. Different modules like Registration, Payments, and Reports are integrated to complete the purpose.

3rd Step- The Scope of Testing

This entire part talks about the functions/ modules in scope & out of range for Testing, any of the items that are not tested because of blockages, dependencies or restrictions.

Like, a functionality verification requires connectivity to a 3rd party app that can’t be tested, as the connectivity was hard to establish because of tech restrictions. This section must be documented appropriately. Otherwise, Testing will be considered to cover each area of the App.

In-Scope:

Functional Testing for the below modules is usually in the Scope of Testing.

Registration

Booking

Payment?

For out of Scope:

Performance Testing didn’t happen for this App.

Items that needed to be tested: Checking connectivity with the 3rd party system or ‘The central repository system did not get tested, as the connectivity was hard to establish because of technical restrictions. This is easy to verify during UAT or User Acceptance Testing, where the connectivity is already there or can be quickly installed.

4th one is the Metrics

This will help you easily understand the results of test execution, the situation of the test cases, defects etc. The needed metrics can be added based on the requirement.

Example: The Defect Summary- based on the severity; defect distribution- function based on the module; Defect ageing etc. The charts or graphs can be easily attached for improved visual representation>

No: of planned test cases vs no: of executed test cases.

No: of test cases that have passed or failed.

?Test Summary Report Number 1.

Test Summary Report Number 2

Number of recognized defects, their status & severity

Test summary report number 3

Test summary report number 4

The module-wise distribution of defects

Test summary report 5

Test summary report 6

5th Step for automation testing reporting

The various types of Testing that’s performed.

Smoke Testing, system integration testing and Regression Testing

Describe in detail the different types of Testing for the Project. This will ensure that the App is properly tested with the help of varying testing types agreed upon according to the test strategy.

Check out these Examples.

?Smoke Testing

This is performed whenever a build is received for Testing to ensure that the maximum functionality is working correctly. The body will then be accepted, and Testing can begin.

System Integration Testing

This is the Testing performed on the Application under test to check the whole working of the Application based on the requirements. The vital business scenarios were tested to ensure that the Application’s crucial functionality works as intended without mistakes.

Regression Testing

Regression Testing was performed every time a new build gets deployed for Testing and consisted of defect fixes and new enhancements.

Whenever?regression testing?is performed on the whole App and not just the new functionality of Defect fixes.

This Testing ensures that the previous functionality works well after fixing the defects and that new changes are added to the last App.

The test cases for new functionality get added to the previous test cases and are executed well.

What skills are required for automation testing other than these? Read more.

6th Step

This is an essential step followed by most software testing companies in India. Share the details for the Test Environment in which the Testing is performed. The server, database, application URL, etc. Some tools, like Quality Center ( currently HP ALM), were used for logging defects.

Eg.

Test summary report 7

7th Step

Lessons that are learnt

Most software testing companies in India believe that this section describes the crucial issues that are usually faced and their solutions (how these were solved at the time of Testing). The lessons learnt will allow you to make proactive decisions at the time of the next Testing engagement by preventing the mistakes or discovering a proper workaround>

Just like,?

Test summary report no: 8

8th Step) Recommendations

<All workaround and suggestions can be mentioned over here>

E.g.,

The Admin control for defect handling tools can be provided to the Offshore Test manager to give access to the?Testing team.

Every time, your onsite Admin is not compelled to be contacted for requests whenever such situations arise, saving your precious time because of the geographical difference in time zones.?

9th Step)?

These are the Best Practices according to most software testing companies in India.

The Testing team will perform different activities at the time of the Project. A few of them could easily save time; some were amazing and proved a fantastic way to work. These can be easily documented for adding value to showcase to the Stakeholders>

E.g.,?

A repeating task performed all the time manually can be time-consuming. This task can be easily automated by making scripts and run all the time. This will save a lot of time and resources. The automated smoke test cases and the scripts are run. This dashed and saved a lot of time.

Automation scripts were made to gain new customers where lots of records should be made for Testing.

Business critical scenarios are tested differently on the whole App, which is essential to certify that they work well.

The 10th Step

< The exit criteria is defined in the form of Completion of Testing by fulfilling some conditions like?

All the planned test cases are implemented;

All the crucial defects are closed.

For example,

The test cases must be implemented- Yes

The defects in essential, primary, and medium severity must be checked and closed- Yes.

If you encounter open defects in extreme severity, the action plan must be made with expected closure dates.

No severity one defect must be ‘OPEN’; Just severity two defects must be ‘OPEN’, and only 4 Severity 3 defects need to be ‘OPEN’.

Important Notice: This might be different for each Project. The plan of action for the open defects must be mentioned appropriately with details regarding when and how they’ll be addressed.

Also, check out our blog on?Why is risk analysis critical in cloud regression testing??

Concluding Words

The Test summary report is an essential deliverable. The focus must remain on preparing a compelling doc, as this artefact will be shared with stakeholders like the senior management, client and all.

After the performance of exhaustive Testing, publishing the test results, metrics, good practices, lessons learned, conclusions on ‘Go Live’ etc., are significant for generating that evidence for the Testing that’s performed as well as the Testing conclusion. For more guidance on automation testing reporting, contact?hikeQA?now!