Test Management - Fully Remote

Wakefield District Housing (WDH)

I joined WDH in June 2020 amid the pandemic and lockdown. The interview was done on Microsoft Teams, something I was used to, having interviewed online using various platforms so it was familiar to me. The brief at WDH was to take ownership of Testing and Releases and ensure new systems, software services and enhancements are fit for purpose.

At the top of my to do list was:

• Review the current schedule of testing in progress & future releases
• Current environment usage
• Review of Azure DevOps open items & bugs
• Implement improvements

The key thing I observed were many open items in DevOps which had been completed and the usage of DevOps. WDH were not using the Test Plan / Test Case functionality & Bugs and instead were using Tasks for everything. I introduced the usage of the repeatable model of using Test Cases and Test Plans and created meaningful dashboards which were useful for Senior Management and Testers alike. I worked with WDH staff to create Test Cases from the tasks, which led to a net saving of 4 person days per D365 monthly release. So now no longer creating 200+ Tasks and instead taking 90 minutes using Query Based Test Suites to introduce reusable test scripts and history attached to each test, based on the current and previous test runs.

This was very successful and illustrates that even when an organisation invests in the latest test tools, without the right guidance can use it in an analogue way instead of the dynamic way it was intended. Before I joined WDH the Testing Dashboard was just a selection of tiles and not really telling you anything except numbers. Nowhere on the dashboard was the current test progress shown or who was doing the testing or which Functional Area the tests were covering. The new dashboard I introduced had all the key things a Test Manager and Tester is looking for: Tests with Pass / Fail / Blocked / No Run status, Test Priority, Test Script Status: Ready/Design. Senior Stakeholders could see a live snapshot which was also sent as part of the daily report and our 3rd party supplier could see the test progress, defect status and defect resolution charts. 

During my time at WDH I trained many staff members on the new (to WDH) usage of DevOps, the main response was this is so much easier, and they all loved the dashboards. I created further dashboards for specific functional areas so they could concentrate on their own area but still had the overall snapshot.

Once WDH had adopted the new way of working I got to work on defect management. I closed down many tasks/bugs which had been delivered and put the process in that any bugs would now be raised as bugs not tasks, and as part of the training material I produced I included how to link their failed test to a new or existing bug.

For a major legacy system release, which had a 7-week UAT split over 3 cycles, I initiated the usage of a Test Readiness Review and twice weekly Defect Resolution meetings. This kept the focus on testing and getting bugs fixed which were blocking tests or key failures and ensured that there was open dialogue and the impact of bugs not being resolved on test progress. By the end of UAT we had achieved an 84% pass rate from 765 tests.

I introduced another concept to WDH of a Defect Retest cycle for the 36 outstanding defects between the end of UAT and Go Live and a further Post Go Live cycle for specific bugs which were Test Environment specific (config or limitations) and once deployed into Production would require a simple regression test. I kept the Defect Resolution meetings in place to again keep the focus on getting bugs resolved. After 2 weeks of the Defect Retest cycle a further 36 tests had passed due to defects getting resolved and the bugs to be resolved prior to go live was down to 10. This was a great achievement for the WDH teams as the quality of testing had improved and led to a higher pass rate and a more granular view of test execution.

WDH had commissioned their partner to create a Web App. This had been in testing and development past the initial agreed go live date and budget by a significant period and was stagnating. I was asked what I could do to assist and get the app over the line. I started by reviewing the agreed Functional Design Document and then created a DevOps dashboard to track testing and defects. I then created a suite of tests to ensure that the app was fully functionally tested based on the requirements which had been given to the supplier.

In parallel I started attending the meetings with the 3rd party and their external supplier. I documented WDH’s issues and bought them to the table and created defects in DevOps which were then sent to the external supplier. On behalf of WDH I managed this supplier and their subcontractor and demonstrated that I understood the app from a technical level and the bugs I was raising were based on the requirements and suggested some usability enhancements. Once the functional testing was completed I did some demonstrations to key WDH staff to show the app had now met the exit criteria from a functional level and suggested that WDH run a limited pilot to get opinion from WDH users. This app was then deployed to the Production environment and the live pilot is now underway.

During my time at WDH I have worked remotely 100% of the time and have not met any of my colleagues in person. This has been a successful demonstration that high-pressure roles can be delivered remotely with the effective use of communication technology, in this case Microsoft Teams, Outlook, SharePoint and Office 365.

My key achievements from this engagement:

• Implemented and maintained testing standards and policies
• Trained WDH staff on the usage of Azure DevOps
• Created and Implemented a defect management policy and managed defects effectively
• Effectively managed internal and external suppliers / developers
• Managed third party risks to an acceptable resolution
• Produced daily, end of cycle, end of stage test reports
• Created and executed test scripts
• Migrated DevOps Tasks and word documents into DevOps Test Scripts and Test Plans
• Managed Test Environments including scheduling
• Took part in a Proof of Concept for test automation with Leapwork
• Provided Test Leadership on multiple projects and releases across Azure Cloud Platform, D365 and tradition Windows Client Server applications

About The Author

Malta Singh is a Senior Test Professional with over 21 years of IT experience. He is an ISEB Practitioner certified Test Manager with experience across many different industries.