The Twenty Laws of Testing Computer Software

As a software development Project Manager, I conduct, plan, organize and cajole the software engineering efforts in companies large and small. During the course of this work, I have never ceased to be amazed at the lack of understanding of both the importance of properly testing a software product or products, and the lack of knowledge around how to correctly conduct the testing effort. QA, or Quality Assurance, is one of the most important aspects of your project's successful delivery, and yet it is the one functional area of your project that is the least well understood in my experiences.

This holds true in corporations both large and small that I have worked for during my over fifteen year professional career. In my opinion, a Project Manager should have a complete understanding of the software testing and QA process, and should also have experience not just scheduling and planning the resources conducting the testing effort, but actual personal testing experience.

It occurred to me earlier on in my career as a Project Manager that in order for me to be a better Project Manager, I was going to have to learn and research everything I could get my hands on about testing computer software and QA. I took courses, I bought books and read them; I related the information I gathered to my experiences as a developer and in some of the ecommerce companies I had worked for and built early on in my career.

I found that this desire to learn the ins and outs of testing was over half the battle towards becoming a more accomplished PM. The Project Manager who appreciates the importance of testing, has been a QA tester, knows and respects the testers on the team, and has a deep seated, fundamental understanding of QA best practices is a Project Manager who commands respect from his project team.

One of my favorite books is “Microsoft Secrets”, by Michael Cusomano. In the book, he describes how early on in the history of the company, testing became a career path on the same level as programming. Knowing, from my extensive reading about Microsoft and Bill Gates, the high altar upon which programmers are placed at Microsoft, I found this to be extremely significant.

Great software development teams and great software engineering companies take the testing of their software seriously. They don’t cut corners, and they don’t have to, because they began with the end in mind.

So without much further adieu, here are my twenty laws for testing computer software. Look for me to expound upon each of the twenty laws in more detail on these pages very soon:

1. The sole goal of testing software is to find errors. Software testing is defined as the method of running a computer software program with the intent of discovering errors in the computer software program.
2. The definition of a good test case is that a good test case is one that has been written in such a manner that it has a great chance of discovery of previously undiscovered errors.
3. A successful test case is one that has been used to discover a previously undiscovered error.
4. Only a high quality software testing process will result in a high quality software testing effort.
5. Testing computer software is a professional discipline that must include skilled and trained professional computer software testers.
6. Someone must assume full responsibility for the improvement of the software testing process.
7. It is vital to foster a 100% positive, inclusive and team-oriented approach with a “test to break” mental attitude.
8. A test case for testing a computer software program must include a definition of the expected result of the computer software program being tested.
9. A computer software programmer should not test the computer software program they have coded themselves.
10. By extension, a computer software programming organization or engineering department should not test its own programs; This is the work of an independent testing organization.
11. The results of each test case should be reviewed with great care.
12. Test cases should be written in order to include unforeseen and invalid user inputs, as well as foreseen, valid user input.
13. Testing a computer software program to insure it performs as it should is only fifty percent of the testing effort. Another fifty percent of the testing effort should be expended in order to insure that the computer software program does not perform in ways in which it should not be performing.
14. Avoid one-time, spontaneous, disposable test cases.
15. A testing effort initiated under the assumption that no errors will be found will not be a successful computer software testing effort.
16. The proliferation of errors in a computer software program can be prevented through the employment of testing during the early stages of the software development lifecycle.
17. Software testing tools can be and should be a key element of a software testing effort.
18. Although perhaps counterintuitive, the probability that more errors will be found in a section of a computer software program in which errors have already been found increases with the number of errors discovered in that section of the computer software program.
19. Testing computer software well is an extremely mentally challenging exercise that requires creativity and perseverance from the testers in order to succeed.
20. The perception (oftentimes forwarded by management) that “not enough time exists to test the product properly, so let’s just ship it anyway”, because the “rewards of shipping the software outweigh the risks of shipping the software with undiscovered errors” may still be common practice in many software development and engineering organizations, yet such an attitude will lead to catastrophe, as software quality is intrinsically linked to customer requirements and customer satisfaction.

Want to know more?

Feel free to reach out to me here on LinkedIn, and you can always check out my former technology blog, Boston’s Hub Tech Insider, a blog stuffed with years of articles about Boston technology startups and venture capital-backed companies, software development, Agile project management, managing software teams, designing web-based business applications, running successful software development projects, ecommerce and telecommunications.

About the author.

I’m Paul Seibert, Oracle | NetSuite #SuiteCommerce Advanced and #Shopify eCommerce consultant here in Southern California (San Diego). Having recently relocated to San Diego from Boston, I have been working in the software engineering and ecommerce industries for over fifteen years. My interests include computers, electronics, robotics and programmable microcontrollers, and I am an avid outdoorsman and guitar player. You can connect with me on LinkedIn, follow me on Twitter, follow me on Quora, even friend me on Facebook if you’re cool. I have a background in entrepreneurship, ecommerce, telecommunications and software development, I’m a PMO Director, eCommerce Practice Director and the co-founder of several ecommerce and web-based software startups. Reach out to me to chat or for advice on Oracle | NetSuite #SuiteCommerce Advanced or #Shopify.