Smoke Testing Vs Sanity Testing

Smoke Testing and Sanity Testing are both types of software testing performed during the software development lifecycle, but they serve different purposes. Here's an explanation of the differences between the two:

1. Smoke Testing:

Smoke Testing is a testing technique performed to quickly assess the stability and readiness of a software build before proceeding with detailed testing. The primary objective of smoke testing is to determine if the critical functionalities of the software are working as expected or if the system is stable enough to undergo further testing. It is typically executed after receiving a new build or version of the software.

Key characteristics of smoke testing include:

• It focuses on high-level functionalities and major features.
• It is usually performed in a non-exhaustive manner, covering a limited set of test cases.
• Its purpose is to identify show-stopping defects that prevent further testing or deployment.
• It is a quick test, usually conducted within a short span of time.
• It helps save time and effort by catching major issues early on in the testing process.

Lets consider an example for smoke testing using Flipkart:

Suppose the development team at Flipkart has made significant updates to their mobile shopping application and provides a new build to the testing team for evaluation. The testing team would perform smoke testing to ensure the critical functionalities of the application are working as expected. They would focus on essential features such as user login, product search, add to cart, and payment processing. If any of these critical features are not functioning correctly, it would be considered a show-stopping issue, and further testing would be halted until the problem is resolved.

For instance, during smoke testing, the testing team might verify that users can successfully log in to their accounts, search for products, add items to the cart, and proceed with the checkout process without any major errors or crashes. This is one of the major functionality of any e-commerce application.

Sanity Testing:

Sanity Testing, also known as a subset of regression testing, is performed when a minor change or bug fix is made in the software. The purpose of sanity testing is to verify that the specific section of the code or functionality has been fixed or modified correctly and does not introduce any new issues. It is usually done after the smoke testing.

Key characteristics of sanity testing include:

• It focuses on specific areas or changes in the software.
• It is a narrow and targeted approach, usually performed on a smaller set of test cases.
• Its purpose is to determine if the recent changes or fixes are working as intended.
• It is usually performed after thorough testing to ensure that the modified code does not cause any unexpected side effects.
• It helps ensure the stability and integrity of the software after changes are made.

Lets consider same Flipkart application for sanity testing example

Suppose a bug has been reported in the Flipkart application related to the product recommendation feature. The development team fixes the bug and provides an updated version of the app to the testing team. Before releasing the fix to the users, the testing team performs sanity testing. In this case, they would specifically focus on the product recommendation functionality to ensure that the recent fix has resolved the issue without introducing any new problems. They would check if the recommended products are displayed accurately based on user preferences and browsing history.

During sanity testing, the testing team might also conduct a quick evaluation of other related functionalities that could be impacted by the fix, such as product filtering or sorting options.

Which one needs to be automated :

In general, both smoke testing and sanity testing can be automated to some extent.

Smoke Testing Automation:

Smoke testing tends to be relatively easier to automate compared to other types of testing. Since smoke testing focuses on high-level functionalities and major features, the test cases are usually straightforward and have a broad scope. The primary goal of smoke testing is to identify show-stopping issues quickly, which often involves executing a predefined set of critical test cases.

We can then Integrate our smoke testing suite with Jenkins configure the SMTP server settings in Jenkins global configuration settings and provide the necessary SMTP server details for sending emails. Then mail will get triggered automatically to the entire team and management after every build. Some organizations refer this process as Health Check also.

Sanity Testing Automation:

Sanity testing can be more challenging to automate compared to smoke testing. This is because sanity testing typically involves verifying specific changes, bug fixes, or updates in a targeted manner. The test cases may vary based on the nature of the modifications and may require more complex test scripts or conditional validations.