Software Testing

Read below some chapters from my Software Testing book published in 2010. Use the Discount Code QPT to buy my ebooks and chatGPT/Gemini course from https://www.rajamanickam.com/

Software Testing Terms

In previous chapters we have learned about importance and some basics of software Testing.

Now we can go into detail about software Testing.

Before going in detail, it is better to have a clear understanding of Terms used in Software Testing.

Find below the meaning/definition of Terms frequently used in Software Testing. Note that I am defining these terms using my own words to explain it easily. It may not be the exact definition. You can refer to any ISTQB documentation if you want to know the exact definition.

Software Development Life Cycle (SDLC)

SDLC involves different phases such as Initial/Planning phase, Requirement analysis phase, Design phase, Coding phase, Testing phase, Delivery and Maintenance phase.

All these phases can be followed one by one linearly as a Waterfall model or they can be followed as a V-model which expects doing Testing activities in parallel with development activities.

Initial Phase involves gathering requirements by interacting with the Customer. Normally Business Analysts will do this and will prepare a requirement document.

Here Customer is the internal marketing team in case of product development. Otherwise Customer is the person who is paying for doing the project.

Requirement analysis phase involves doing detailed study of the customer requirements and judging possibilities and scope of the requirements. And it involves tentative planning and technology & resource selection.

SRS (System Requirement Specification) will be created in this phase.

Design phase involves dividing the whole project into modules and sub-modules by doing High level Designing (H.L.D) and ? Low level Designing ? (L.L.D).

Coding Phase involves creating source code or program by the programmers by referring to the design document.? Coding standards with proper comments should be followed.

Testing Phase involves getting clarification for the unclear requirements and then writing test cases by the Testing Team based on the requirements. And, the testing team will execute the test cases once the build is released and they will report the bugs found during the test case execution.

Delivery & Maintenance phase ?involves installing the application in the customer place and providing the details such as release notes to the customer.

The Maintenance or support Team will help the customers if they face any issue when using the application.

Software Testing - It is the process of verifying whether a software application or a software product meets the business and technical requirements. i-e verifying whether the developed software application works as expected.

It will be done by comparing the Actual result against the Expected Result.

For example, if the requirement says “After entering valid username and password the user should be logged in to the website” software testing is nothing but just entering valid username and password to verify whether the user got logged in to the website.

But, doing software testing is not so simple, there are a lot of complex factors involved in doing testing. It needs a lot of thinking and brainstorming processes.?

In this simple example, we should know below things.

-In which environment we need to do the login testing?

-What is valid username and password?

-How to verify whether the user really logged into the website?

-How long it will take to complete the login?

-How the system should behave if the user enters invalid username or invalid password?

-What will happen if the user is already logged in?

-How the system should behave if someone continuously tries to login by giving invalid username and password?

-Is it possible to reach the target page even without doing login?

-How the system will behave if there is any network issue once after entering the username and password?

-What will happen if two people try to login with the same username and password simultaneously from two different machines?

-How the system will behave if the user doesn’t use the website for longtime once after getting logged in?

-Will the login process take the same time even after Millions of users registered with the system?

-What will happen if thousands of people try to login at the same time?

-What message will be shown to the user if the database which stores the user details goes down?

- Will the system provide any other interface (e.g API, webservice) other than the standard UI for doing login?

-How to help the user if he forgot the password?

-Will the system allow the username in non-English also?

-Most importantly, when do we need to do this login testing?

-What are the pre-requisites? (e.g user should be registered before testing login).

-Is it necessary to do the testing in production environment also?

-To whom should we report if the Testing fails? And, what are the details we need to provide? How do we come to know once it got fixed? What do we need to do once it 's fixed?

-Is there any option (e.g Remember Me cookie, auto-complete) for making the login easy? Will these options break the login system in any way?

- Whether anyone (e.g hacker) can access the password while doing login?

-Whether the password is not readable to the developers of the system so as to avoid any misuse? (e.g password encryption before storing it in Database)

-If we need to do this testing multiple times in many environments, is there any easy way to do the testing? (e.g using scripts and automation tools such as QTP)

-Will it work correctly in different user interfaces (e.g Different browsers such as Internet Explorer, FireFox, Safari, Chrome), and in different settings (e.g javascript disabled)?

-Is the code written well enough so that future enhancements can be made easily??

-Who has to do this testing? Whether the developer of the system can do the testing also?

?

So, it is important to have a clear understanding of testing approaches before start doing any testing.

Functional & Non-functional Testing

Functional testing mainly focuses on verifying whether the features requested in the requirement document are working correctly.

Non-functional testing is checking the performance, stability, scalability, usability, internationalization and security of the software application.

Testing methods

Whitebox, blackbox and Greybox are three Testing methods.

White box testing will be done by going thro’ the coding and by understanding the algorithm used in the coding.? It includes API testing and code coverage.

Black box testing will be done without knowing internal structure or coding of the application. It will help to find more bugs effectively. But the tester may spend more time by writing many test cases to check something which could have been tested easily by writing one test case.

Grey box testing involves having knowledge of internal data structures and algorithms for writing the test cases, but testing at the user, or black-box level

Testing levels

Unit Testing or component testing will be done by the developers to make sure that the small piece of the code works correctly.?

Each and every unit of the program will be tested in order to confirm whether the conditions, functions and loops are working fine or not.

Integration Testing will help to expose defects in the interfaces and interaction between many different modules of the program.

System testing will be done by the Testing Team to make sure that the program or application meets the requirements.

It includes GUI software testing, Usability testing, Performance testing, Stress testing, Security testing, Scalability testing, Sanity Testing, Smoke Testing, ad hoc testing, etc..

Regression testing will done to make sure that the application or program is not affected by any code change done to the application. i-e Already working functions in other modules of the program should continue to work after changing any of the module .

We need to test each part of application even when the code change is done in any specific part or module of the program. Automation tools will be useful for doing regression testing.

Alpha Testing will be a part of user Acceptance testing.? It will be done in the developers premise, and will be done by the customers or by independent test team.

Beta testing comes after alpha testing. Beta versions of the Software will be released to a limited number of people outside of the programming team.

Actual release will be done if there are no major issues found in Beta testing.

Testing Artifacts

Test plan is a document which describes the objectives, scope, approach, and focus of a software testing effort. It will be made available for development team and business people also so that they can understand the testing activities done by Testing Team.

It will cover the features to be tested and features not be tested.

Testing environment details, risks, Responsibilities, testing schedule, Test deliverables and resource allocation details will be included in the Test plan.

So, it will be useful to have overall view of the testing activities to be done in particular release of a software application.

Traceability matrix is simply a mapping between the requirements and the Test cases.? It will be prepared in a tabular form. i-e in excel spread sheet.

Once column will have the list of the Requirement IDs and the next column which have Test case IDs which test that requirement.

It will help to make sure that test cases are written well enough to cover all the requirements.??

Similarly we can have reverse Traceability matrix also. i-e mapping between test cases and the requirements.

It will help to make sure that we are not having any test cases for the requirements which are not asked by the customer.

Test suite will be the collection of the Test cases. Mostly all relevant test cases will be grouped as one test case document. For example, the test cases which will test the login module will be stored in a particular spread sheet file named as “login_testcases.xls”. It can contain information such as Name of the module, description, total number of test cases and details of reference document (i-e requirement document, use case,etc).

Test case will have below things.

????Test case ID for uniquely identifying the test case. For example, Test case ID can be TC001,TC002,….

????Test case description will have condition which we are going to test.?

e.g To verify user sees the message “invalid login details” when they enter valid username and invalid password”

????Test steps will give details or steps required for executing this test case

e.g 1. Go to the login page

??????2. Enter valid username.

??????3. Enter invalid password.

??????4. Click “login” button.

????Expected Result will give the details about the behavior or result we should see once after executing the test steps.

e.g User should see “invalid login details” message in Red color at top of the page.

????

????Author who writes this test case.

????Automatable- To mark whether this test case can be automated using automation tools such as QTP.

Apart from above things we can add pass/fail and remarks while executing the test cases.

Test case can be written by referring use case document and requirement document. We may need to refer the application for writing test cases.

We can use some techniques such as Equivalence partitioning and boundary value analysis for writing test cases.

According to Equivalence partitioning, writing one test case for each partition of the input data is enough.

For example, if a password field accepts minimum 4 characters and maximum 10 characters, then there will be three partitions. First one is a valid partition 4 to 10. Second is invalid partition of values less than 4. Third one is another invalid partition of values more than 10. We can take one value from each partition to do the testing.

In this example the boundary values based on boundary value analysis.

are 3,4,5, 9,10 and11.

Software Test life cycle.

Test Planning - Scope of the testing will be defined according to the budget allocated for the testing. And, Test plan document will be prepared by the Test manager.

Test development- Test Cases will be written by the Testing Team (QA Team) during this phase.? Test data files also will be created.

Test execution- Testers will execute the test cases and will report the issues to the development Team for fixing them.

Performance test should be executed only after the functional and regression testing got completed.

Bug Tracking is the methodology used to follow up the bugs/defects/issue found during Test execution. There are many free tools (e.g Bugzilla.) available for doing bug tracking effectively.

Normally the bug will be tracked as standard bug life cycle.

It will have below states.

New: When a tester finds a bug first time the state will be “NEW”. This means that the bug is not yet approved.

Open: After a tester has posted a bug, the lead of the testing team will check whether the reported bug is genuine and the he will change the state as “OPEN”.

?Assign: The developerment Team lead will assign the bug to particular developer for fixing it. Now the state will be changed to “ASSIGN”.

Ready-to-Test: Once the developer fixes the bug, he will assign the bug to the testing team for next round of testing with the status "READY-TO-TEST".

?Deferred: The status will be changed to "DEFERRED" if the team decides to fix it in next release or the priority is very low.

Rejected: If the developer decides that the bug is not genuine, he can reject the bug. Then the state of the bug is changed to “REJECTED”.

Duplicate: If the bug is repeated twice or the root causes for two bugs are same, then one bug status will be changed to “DUPLICATE”.

Verified: Once the bug is fixed and the status is changed to “Ready-to-test”, the tester tests the bug again. If the bug is not present in the software, he approves that the bug is fixed and changes the status to “VERIFIED”.

?Reopened: If the bug still exists even after the bug is fixed by the developer, the tester will change the status to “REOPENED”. The bug will go thro' life cycle once again.

Closed: Once the bug is fixed, it will be tested by the tester again. If the tester verifies that the bug no longer exists in the software, he changes the status of the bug to “CLOSED”. This state means that the bug is fixed and tested again.

Reporting- Test summary report should be created for explaining the steps taken for delivering quality product. This summary report should show the number of test cases executed, how many passed and how many failed, test coverage, defect density and other test metrics.? And, it should show performance test result also.

1. Writing Good Test Cases and Finding Bugs effectively

To develop bug free software application, writing good test cases is essential. Here, we will see how to write good test cases. Before seeing this, we should understand what is Good Test Case. There won't be any solid definition for "Good Test Case". I will consider a Test Case as "Good" only when a Tester feels happy to follow the steps in the Test Case which is written by another Tester. Test Cases will be useful only if they are used by the people.?

If a test case is poorly written with excessive unwanted steps, then most of the Testers won't read it fully. Just they will read few lines and will execute it based on their own understanding which will be mostly wrong. On the other hand, if it is having fewer details then it is difficult to execute it. As of now, I am thinking below things for writing effective Test Cases.

• Before start writing test cases, become familiar with the (AUT) Application Under Test. You will become familiar with Application by doing some adhoc/exploratory testing.

• We should read the requirements clearly and completely. If we have any questions in the Requirements it should be clarified by appropriate person (e.g Customer or Business Team). And also, it is good practice to gather some basic domain knowledge before getting into reading requirements and writing Test Cases. And also, we can have discussion/meeting with developers/business team

• Very Important thing is, we should use only simple language or style to write the Test cases so that any one can easily understand without any ambiguity

• Give meaningful and easily understandable Test case ID/number. For example, if you are writing Test case for testing Login module you can Test Case ID as below. 1a - for testing positive scenario such as entering valid username and valid password. 1b - for testing negative scenario such as entering invalid username and invalid password. By giving Test Case number as above instead of giving sequential number, we can easily add any new case such as below one without needing to adjust/modify number of any other subsequent test cases. 1c- for testing negative scenario such as entering valid username and invalid password.
• And also, if we have any similar module we can give separate sequence number for specifying the module. For example, assume that we are having separate login modules for User and Admin with little changes. In this case we can give number as below, 1.1-First case in User module. 1.2-Second case in User module. 2.1-First case in Admin module 2.2-Second case in Admin module. If Test Description/Steps/Expected Results of 2.1 is mostly same as 1.1 then we should refer 1.1 in 2.1 instead writing the sample details again. By doing like this, we can avoid redundant details to have clear test case document.

• Test Description should be short and it should uniquely represent the current test scenario without any ambiguity.?

• In any situation, don't use "if condition" in the Test steps. Always address only one scenario in one test case. It will be helpful to have unambiguous Expected Result.?

• Give some sample test data that will be useful for executing the test cases.?

• If the Test Case requires any Preconditions/prerequisite don't forget to mention them. The best way is, we should arrange/order the Test Cases such that the need for specifying precondition is minimum. For example, we need to write test case for testing user creation, user modification and user deletion. For doing user modification and user deletion we should have already created user as precondition. If we arrange the Test cases in below mentioned order, we can avoid the need for specifying any preconditions/prerequisites. 1-Test case for creating user. 2-Test case for verifying duplicate/existing user when adding another user with same username. 3-Test case for modifying user. 4-Test case for deleting user.

• Keep Traceability Matrix to make sure that we have written test cases for covering all requirements.

• Once after completing all positive scenarios, think about all possibilities of negative scenarios to have test cases which will effectively find most of the bugs. For doing this we can refer alternate flow section of use case document, and we can think about different data, boundary conditions, different navigations paths and multi user environment.

• In the test case document, we can give link to screenshots explaining the steps and/or expected results with pictures. But anyway, it is not good practice to place the screenshots within the Test Case document itself unless it is very essential?

• Many tools are available to capture the screenshots with user action as video. We can make use of them to keep video explaining the steps and expected results clearly in case the test case requires any complex steps. We can give link to this video from the test case document.

1. Tips and Tricks for doing AdHoc Testing

It is always not possible to follow proper testing such as writing Test Plan and writing Test cases. In some cases we may need to go with adHoc Testing because of time constraint or resource constraint. AdHoc Testing is the part of Exploratory testing. It is done without doing Planning and Documentation. Adhoc testing will help to find the defects earlier. We know that earlier a defect is found the cheaper it is to fix it. Here I am listing some tips for doing adhoc testing effectively.

• In case of UI (User Interface) testing, test all navigation including Back button navigation. Go thro' all the pages of the application to find any broken links and also make sure that each and every page is having proper links to reach other pages either directly or indirectly. And, notice the page loading time also. If it takes more time then try to narrow down whether it is due to network issue or web server issue or? the page issue.? i-e If the other domain webpages are loading normally then we can say that it is not due to network issue. And, if other pages in the same domain are loading normally then we can say that it is not related to server issue.??

• Check whether all the images are having alt attribute. And anchor tags should have title attribute.

• See the application screen or webpage by changing/setting different screen resolution in your computer monitor.

• Test the webpage in many different web browsers such as Internet Explorer, FireFox, chrome, safari, etc.?

• Test the tab order and default focus in all the pages. Especially study the behaviour of the application when it has more than one submit button.

• Try to enter/save test data having special characters such as single quotes, double quotes and comma .

• You can try to enter text with HTML tags such as < and > also in the textbox?

• Try to load an authenticated webpage directly by entering url in the browser without doing login.?

• Try all the possibilities of boundary values such as entering lot of data in textbox and entering negative values in numeric fields.?

• Remember to navigate the application from two different machines/browsers simultaneously; especially concentrate on to test concurrent database saving/updating operation.?

• If possible/necessary, test the application in different OS (Operating System) such as Windows, Linux and MAC.

• If your webpage uses flash files, try to see the behavior of your webpage when it is loaded in a machine which is not having flash player. Normally it should automatically download the flash plug-in.?

• Instead of testing everything from your local machine, just try to test some screens by hosting your site in some remote machine. It will help to identify unexpected issues which may occur due to network latency.?

• Test Session timeout, Cookie expiry and script execution timeout.? For testing these things you should know about session and cookie. Session variables are stored in server side. They will be unique for each browser session. It will expire based on server setting. Cookie will be stored in user browser. It will expire based on the expiry time set while writing the cookie. But the user can manually also delete this cookie from the browser before it actually expires. Since user login status is mostly controlled using session and cookie variables, it is very important to test behavior of the system when the session or cookie expires.

• Try to refresh your confirmation screen many times to verify whether the multiple refresh saves/inserts the data multiple times.?

• Test with different Date and Time format if you webpage/application has date and time entry fields. And, think about Time zone also.?

• Make sure that Number/Currency/Name format is correctly displayed in all pages uniformly.

• When testing edit/modify/update feature, modify values of all fields and make sure that everything is getting updated correctly.

• Whenever testing Delete feature make sure that all the related data also getting deleted. For example, when deleting question and answers also will be deleted. And, make sure that necessary constraints are enforced correctly. For example, deletion of questions should not be allowed if the questions are already used in some other modules.

1. Best practices in Software Testing

There are lots of materials available in internet to explain best practices in Software Testing. Here I am writing only the very essential things for medium level projects based on my experience/view point.

• We should start our testing activities at beginning of Software development itself. Understanding Scope/purpose of the project will help to judge the degree/level of testing required.

• Testers should go thro' the requirements in detail without missing any points given by the client before writing test cases.

• The test cases should be updated immediately once the client gives new requirement or changes the requirements.

• The test case document should cover all the requirements even if some requirements are non-testable. These non-testable items should be marked as non-testable. Keeping traceability matrix document will helpful to achieve this.?

• The Test case document should help to clearly identify hierarchy/arrangement of test cases. It should have clear approach to arrange test cases if many test cases exist with similar steps. It is not advisable to copy & paste the similar test cases many times, instead we can specify only the additional/different steps.?

• Description of each test case should be written clearly after understanding the context/module of description. Steps should be written only after manually executing them. Expected results should not have any ambiguity. If required, Prerequisite/preconditions should be mentioned.

• Planning and creating test plan document is essential even for small short-term projects. The test plan document need not contain all the details, but it should contain at least very basic components such as scope, schedule, risks, environments, testers?

• Planning of development/test/staging environments should be done clearly. And it is very important to move the code and maintain version of code in each environment without any ambiguity/confusion. Testers should know which version of code/data is available in each environment?

• Test execution should be done carefully based on the test cases. It is very important to use appropriate test data. It is better to create different set of test data during test case creation itself. The test data should cover valid format, invalid format and boundary values. Test result (pass/fail) should be clearly updated for each test case. It is good practice to mention Actual behavior if the test case fails. The test results should be communicated to the other parties (developers, business/client) daily even if all the test cases are not executed. In this case, we should add a note to indicate that the test execution is still in progress. The test execution summary document/mail should clearly mention date of execution, environment, test name and test result.

• In case, most of test cases are getting failed continuously, there is no meaning of continuing the execution. Execution should be resumed once after fixing the major issues.?

• It will be nice if we highlight the testing status (pass, fail, yetToStart) in appropriate color. But anyway, just highlighting the test case with appropriate color without specifying status is not a good practice. Because while taking single color printout of the test report, it is difficult to see the status from the color.

• It is good practice to do some adhoc testing in addition to the test case execution.

• Clear/proper communication/co-ordination within the Testing team and also with other teams (developers, client/business) is very essential.?

• The bug report should be prepared very clearly with all essential details, especially with the steps/testdata for reproducing the bug. The bug report should help the developers to reproduce the bug and to fix it.?

• Doing re-test and small regression test is essential whenever a reported bug is fixed

It is not good if we do all the testing manually, as manual testing will take more time/effort and it is difficult to manage, and also it not consistent or repeatable. So it is better to automate the test cases using test tools. Even we can use simple shell scripts and vbscript to automate some part of the testing.

These are some chapters from my Software Testing book published in 2010. Use the Discount Code QPT to buy my ebooks and course from https://www.rajamanickam.com/

Contact me ([email protected] ) for hiring me to do PHP web development, Software Testing and Digital Marketing. Currently, I charge $6 per hour.