Bug Process & Power of QA (Beginners Guide)

INDEX

 

 

1. Upcoming Project discussion and collection of documentation……………..3
2. Discuss & Decide Criteria OR Phases of the testing……………………………..3
3. Classification of Application Modules………………………………………………….3
4. QA Server(s) …………………………………………………………..……………………..3
5. Build Number………..…………………………………………………………..……………3
6. Bug submission and process……………………………………………………………..3
7. Before you report a bug…………………………………………………………………….4
8. Report a bug……………………………………………………………………………………4
9. Power of QA…………………………………………………………………………………….9
10. After submitting new bugs………………………………………………………………..5
11. Set a priority……………………………………………………………………………………5
12. Standard bug resolutions…………………………………………………………………..6
13. Standard bug status………………………………………………………………………….6
14. Bug Process Diagram………………………………………………………………………..7
15. If it is Not-Reproducible…………………………………………………………………….8
16. If it is Duplicate………………………………………………………………………………..8
17. If it Won't Fix……………………………………………………………………………………8
18. If Need More Information…………………………………………………………………..8
19. If it is Invalid…………………………………………………………………………………….8
20. If it is Crazy Talk……………………………………………………………………………….8
21. If it is As per Design…………………………………………………………………………..8
22. Resolved ticket………………………………………………………………………………….8
23. If resolved ticket is passing………………………………………………………………..9

    

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Upcoming Project discussion and collection of documentation

 

Discuss the project definition and current status of the project (whether it is in pipeline, will start work in nearest future OR it is in Development phase) and keep communication with the PM/Dev/Client and collect and reading of documentation and report regularly to HOD.

 

Discuss & Decide Criteria OR Phases of the testing

 

After commencement of the project in QA, discuss about modules available in the project and how critical it is. Sometimes in some cases single project may have different application and its modules, so it is preferable to discuss about all modules and required QA efforts with the PM.

 

You can identify efforts required on a base of reviewing RDS (Requirement Document Specification) and if application is already exist on a QA/Dev/Production server then it would be very helpful to you to analyze total efforts required.

 

Base on above paragraph, estimate a man hours and report to PM regarding your estimation, if timeline is near than estimated hours then discuss clearly with PM, in this situation two things can be happen

 

1. either provide extra man power OR
2. filter out(decrease) criteria of testing

 

and discuss both the criteria with the PM and if he is not agree with any of above two then alert him about drawback of this type of testing and QA will be not responsible for any kind of issues later.

 

In some cases you can save your time by minimizing a work on a test case writing, if test cases is not hardly needed then just create a checklist of possible test scenarios and execute it while testing.

 

Classification of Application Modules

 

Classify different modules of the application and prioritize them according to their seriousness in the application and then start your next step of the testing. Even if application is a data driven application then also create a check list of test scenarios with priority & flow of the modules.

 

QA Server(s)

 

Be sure about one important thing is that, always test on QA server and not on Development or not on production directly, QA should have or can demand for the separate QA server, and it would be great if QA can have a multiple QA Servers for the comparison purpose so that you can compare any critical issues on previous build also which is available on another QA server.

 

Different QA server can have a different builds for the comparison purpose so that QA can identify original build of the issue, and can make sure that on which build issue was exist.

 

Build Number

 

When applicable, build numbers should be used to specify what version of application a bug was found in, what version it was fixed in, and what version a fix was verified in. Revision numbers of source code modules modified to fix bugs are helpful notes for developers, but they aren't necessary or sufficient to coordinate work between test and development. Use build numbers instead.

 

Bug submission and process

 

After all clearance about project and release dates, start first testing cycle with or without test cases as per availability and requirement of test cases.

 

Before you report a bug

 

With large projects, so many users report a bugs than there's a good chance that your bug may have already reported. Because of this, it's very important to check to be sure it's not already in the system (Bug Tracker) before you submit it. If you are new to reporting bugs, it is also a good idea to discuss the issue with more experienced QA or Developers before reporting it. Please follow the steps below.

 

It is always good to find out a root cause of a bug and reason of the bug, it may be some times due to incorrect requirement or environment from the client OR it may be a reason of some malpractices performed by QA himself.

 

Search for your bug or feature request in Track by using Search or a Query.

 

If your issue has already been reported, please do not report a duplicate bug. If you have further information to contribute, log in and add a note to the existing bug.

 

If your issue is similar, but not quite the same as another issue, you may decide whether to add a note to the similar issue, or report a new one. It can be difficult to decide whether your issue warrants a new submission, but in general if you just have more information to Contribute to a current, open issue; simply add a note to that issue. If you have a different enough issue, or if you are experiencing a recurrence of an issue that was previously resolved, report a new bug.

 

If your issue was recently reported and then closed, and you do not agree with the resolution, you can also reopen the ticket and add a comment as to your reasoning.

 

It is best not to re-open bugs that have been closed for some time (E.G. On a milestone relating to a version more than 1 release old) - in that event consider a new ticket.

 

Please do not change the version number on bugs that have been reported already. The version number relates to the version in which the bug was originally discovered. If you're seeing the same bug in a newer version, mention so in a comment.

 

 

Report a bug

 

During the test case execution if you are finding out any issues then make sure about an issue and reproduce it multiple times with different environment and also try to reproduce with different steps and on different browser type and versions, then create a bug ticket in a bug tracker with a valid and easy way so that other QA or Developers can identify and understand your ticket submitted.

 

Sometime it happens that QA person submitting issue in short description and summary for e.g. If any layout issues is there then some QA person will right like “Layout issue” only and will provide only screenshot in a description, in this case it may be very hard to identify exact place of issue and the way to regenerate.

 

Make the summary short but informative and accurate, as this is the ticket title that will be displayed in search results.

 

Always mention Module Name then particular page name then problem and how it happen, in a summary and if project has multiple applications then also include application name in beginning of the summary (so thus your summary will be divided into 4 parts), and describe it fully in a description and don’t forget to provide screenshot if possible.

 

Example.

 

LMS – Apply Leave – Unable to apply a leave with valid date formate

 1              2                          3                              4

 

in above example 1st is indicating name of an application, 2nd is indicating module of application, 3rd is indicating exact problem what is happening in application and 4th is indicating the reason of problem that why it is happening and in a description.

 

And describe the problem with full details.

 

Example.

 

Found for the package [Package No.] on QA[Server Name or No.]

 

e.g. Found for the package 2.0.0 on QA1

 

Browser Used: EI8

 

Go to LMS – Apply Leave –click on Apply New Leave button,

Fill up required fields, and in a date field enter date manually or select from calendar control (i have selected '01/02/2011 ') and press submit button. Observe that it is displaying following error message.

 

Error: XYZ

 

Refer screenshot for more visual details.”

 

during submitting a new bug, select related categories and assigned to responsible person or Developers.

 

Above is the general and easy way to submit a issues in a bug tracker, and dont forget to include Package OR Build Number, QA Server Number(OR Name) and Browser type with the version and User credentials you have used for the testing if possible, because it will be great help to the developers and other QA to identify problems in a easy way.

 

While attaching the screenshot, don’t forget to crop title bar of your screen as it looks very unprofessional and never crop address bar because it can be help to identify proper path to the Developers or other QA person, highlight effected areas in a step way by the help of image editing tools and write down a problem if possible.

 

After submitting new bugs, now it is in Open status and if developer cannot reproduce those particular issues or facing any problem or want any clarification then he should write down appropriate comment with sufficient details in the same issue and should assign back to Particular QA so that QA can revert back to him with proper details.

 

Now if QA cannot reproduce the same issue again himself with the possible steps then he should close the same issue with the resolution not-reproducible and status closed, and should write down the comment like “Unable to reproduce for the Package [Package No.], closing as invalid.”

 

Set a priority

 

Priority either can be a number of 1 to 5 or 1 to 10 OR can be like High, Medium and Low in a different bug tracker.

 

Set appropriate priority as needed during the submitting a bug

 

Bug Severity

 

Severity is a seriousness of the issue, it can be as below.

 

Blocker

Critical

Major

Minor

Normal

Trivial

Enhancement

New feature

 

etc.

 

Set a Version

 

This version will indicate that in which build issue was found or exist.?

 

Set a milestone

 

Milestone is a build no. Which will be available in a future and it indicates that in which build this problem will be solved.

 

Standard bug resolutions are as below.

 

Not-Reproducible

Won't Fix

Fixed

Duplicate

Need More Information

Invalid

Creasy talk

As per design

 

Standard bug status is as below

 

Open

Resolved

Closed

Re-open

 

We can also use more status for easy understanding, like...

 

Failed-Minor

Failed-Major

Completed

Obsolete

Etc.

 

During the ticket submission, some small things can be very effective like a font size and style, maintain font size and size (standard Verdana, 10px) during each bug submission.

 

Avoid using a committee or meeting to disclose bugs reports. This will burn a lot of QA man hours that could be used fixing bugs. If the schedule dictates a triage, don't close bug reports in the meeting because this will circumvent the quality process by sidestepping testers that report bugs. Use the bug triage to reprioritize, postpone, or resolve as won't fix instead. And keep each track record of communication you have done with every related person.

 

Avoid unnecessary discussion with the developers and make habit to communicate for any issue point via comment in each ticket instead face to face conversation.

 

 

 

Bug Process Diagram

 

 

 

 

 

 

After submitting a bug there are long process to reach at the stage of close an issue.

In between issue can have a different stage or status while the process.

 

We will see different stages with status and interaction between QA and Developer during the bug in process.

 

 

 

If it is Not-Reproducible at the developers end, then he should commented properly that issue and assigned back to author QA with the request for more clear steps to re-generate particular issue.

Here two different situations can be take place.

1. If it is not reproducible then developer can Close that issue as Not-Reproducible OR
2. Developer can assigned to QA with request for more clear steps

 

If second situation take place than either QA should paste proper comment with the more clear steps if issue is really re-producible at QA's end OR if it is not re-producible then he should Closed the issue as Not-Reproducible and don’t forget to mention build number and QA server on which this is not reproducible.

 

If it is Duplicate then Developer can directly close this ticket as Duplicate resolution and also add a Original Ticket number in a comment and if QA found that submitted ticket is duplicate then find out an original ticket number, include it in comment and close it out with Duplicate resolution.

 

If it Won't Fix then Developer can close it out as Won't Fix with the appropriate comment and also should add detailed comment in it which can highlight that why it will be not resolved or fixe what is the reason that he is putting it as Won't Fix.

For Example

Generally When some functionality has an issue which is very minor and not effecting any functionality or business logic and layout but it requires much time and process to resolve it then it can be put as won't Fix.

 

Fixed is used for fixed issues which has been resolved now.

 

If Need More Information this resolution can be used in many cases, if there are multiple QA is available and the people who is testing currently this issue is not aware with the functionality or requirement of the issues then he can set this resolution with the appropriate comment and request for more detail about this ticket and assign back to author QA or Developer. And if developer is not aware with it then he can also set this resolution and can assigned back to author person.

 

Thus Author person will add more detail comment to the ticket and assigned back to requested people so that he can go ahead with it.

 

If it is Invalid then developer should paste detailed comment in it and close it as invalid and assigned back to Author, and if possible then please include the ticket number of requirement so that author person can know that why it is invalid and what he did wrong with the functionality.

 

If it is Crazy Talk then close as crazy talk OR you can also use Invalid resolution here, crazy talk can use only if in such cases submitted issue has no any logical base or it is a just foolish or bizarre things wrote in a ticket, and make sure to avoid any personal comment as it looks unprofessional.

 

If it is As per Design then developer OR other QA should close this issue with the resolution 'As per Design' and assigned back to author with appropriate comment and also include reference of requirement document or improvement ticket number.

 

Resolved ticket can be test only in next build and before the start testing always check a milestone build number of the ticket so that you can decide that in which build you have to test this particular ticket.

If resolved ticket is passing then close it as Fixed and don’t forget to include build number and QA server identification into the comment.

And if it is Failed then Re-Open that ticket and assigned back to the developer who has fixed this issue, and also mention proper steps to regenerate that issue and additional information, build number and QA server if any.

 

Sometimes it happens that problem has been not resolved completely but partially it has been resolved, at that time we can use such resolutions like “Failed minor” or “Failed Major”.

 

Be a Good Tester

 

It is always

 

Power of QA

 

After all bug process, if build has been ready to release then PM is responsible to inform officially to QA via email about release and he should give a date and time for build release (release date should be pre decided) whether it is on QA server or on production server. And if QA is not ready for the build then QA should give a clear reason and the list of the areas has been tested and has been remains to test. It is unofficial to release a build without inform to QA and QA is not responsible for the any matter afterwards.

 

If it is on production server then there must be schedule of smoke test on stage and production server with the specific checklist and in presence of each related person (QA, Dev, Operation people, PM, QA Manager) release should be take place. And if during smoke test any crash or blocker issue come across which is in sensitive area of an application then in any situation QA should not allow OR Green light to the release and QA manager or HOD has rights to stop a build to be released.

 

If problem can be fixed at a time by developer and it is not much critical to solve for the developer then it is preferable to resolve issue on the spot and for the same also bug ticket should be created by bug founder and after resolving the problem it should be closed if it passes.

 

Thus QA has a power to stop a build release if there is proper reason and without permission or positive report from the QA, no one has to rights to release a build on a production and if it is happening then QA is not responsible for any upcoming problems.

 

 

 

*I will appreciate your suggestion/Correction and comments on this article*

 

 

Author:

Sajid Y. Sidi

Software Test Lead