What is smoke testing?

Smoke testing, also called build verification testing or confidence testing, is a software testing method that is used to determine if a new software build is ready for the next testing phase. This testing method determines if the most crucial functions of a program work but does not delve into finer details.

As a preliminary check of software, smoke testing finds basic and critical issues in an application before more in-depth testing is done. If a piece of software passes a smoke test, quality assurance (QA) teams then proceed with further testing. A failure means the software must be handed back to the development team.

The goal of smoke testing is to discover simple but severe failures using test cases that cover the most important functionalities of a software. Smoke tests are performed by QA teams using a minimal set of tests on each build that focuses on software functionality.

Without smoke testing, major issues could slip through the cracks, leaving the chance for those issues to cause larger problems down the line.

The term smoke testing originates from a basic type of hardware testing in which a device passes the test if it doesn't catch fire the first time it turns on. Daily build and smoke tests are among industry best practices advocated by the Institute of Electrical and Electronics Engineers in Code Complete by former IEEE editor in chief and software engineer expert Steve McConnell.

What are the different types of smoke testing?

Types of smoke testing include the following:

• Manual smoke testing. Human software testers conduct smoke tests manually. This includes manually developing and updating test cases. Test scripts are also written manually for new or existing features.
• Automated smoke testing. Software tools are used to automate the smoke testing process. Smoke testing tools make the testing process more efficient by automatically providing relevant tests.
• Hybrid smoke testing. Hybrid testing is a combination of manual and automated smoke testing, where testers write test cases and then automate the tests using a tool.

How smoke testing works and how to perform a smoke test

QA testers perform smoke testing after the developers deliver a new build of an application. The test is performed manually or automatically using previously written scripts.

If the smoke test passes, the software is integrated with an existing build in the QA and staging environment. The software build then moves on to more rigorous tests, such as unit and integration tests. If the smoke test fails, then the testers have discovered a major flaw that halts all further tests. QA teams then ask developers to send a fixed build.

This broad initial test is a more effective strategy to improve software code than if the team conducted specific and rigorous tests this early in the software development process.

Smoke testing is also performed from the perspective of the user experience. This approach includes testing key functionalities, such as if the build is accessible or if the user interface and login mechanism function correctly. Other key functionalities include if a selection action correlates with the intended action.

How smoke tests are run depends on the type of smoke test, and approaches may vary from one organization to another.

A general process for performing a smoke test may look like this:

1. Decide between manual, automated or hybrid testing. Manual testing is most commonly chosen first, while automated tests may be done once more elements are added to the smoke test.
2. Create testing scenarios. QA testers need to know how many test cases they need for each core software feature. Pass or fail metrics should be decided on based on software and organizational needs and standards.
3. Create the smoke tests. Either using manual or automated processes, write and create the test suites.
4. Run and record tests. While smoke tests are run, testers should have a process set to record the results for each test. This can either be manual or automated.
5. Analyze smoke test results. If the software fails, it's returned to the development team for more testing. If it passes, it is ready for more rigorous functional testing.

Manual smoke tests require the testers to write and update smoke tests. Scripts are manually modified to fit each requirement. In automated smoke tests, additional tools are used to test core features quickly. Hybrid smoke tests have a tester write the test cases, while automating the tests using smoke testing automated tools.

Advantages of smoke testing

Smoke testing is a valuable part of the testing process for the following reasons:

• Automation enables tests to be run faster and more often.
• Future builds are constructed on bug-free software.
• It identifies defects in early stages of development.
• It lessens risks associated with adding code to existing build integrations.
• Smoke tests can be run often due to their low resource requirements and simple process.
• There is flexibility in implementation with manual, automated and hybrid types.
• It verifies software quality early on in development, ensuring that a build is stable.

Disadvantages of smoke testing

There are some disadvantages that come with smoke testing, however, for example:

• Smoke testing doesn't cover all functionalities in a piece of software -- only specified core functionalities are tested.
• Not every bug or problem may be found with smoke tests, and some may still be found later in development.
• Manual smoke testing takes longer to complete when used in larger projects.

What is the smoke testing cycle?

The smoke testing cycle starts with a build being delivered from the development team. That build is passed for an initial smoke test by QA. Specific test cases are prioritized based on core or key features, tests are created and the software is then tested.

If the software fails the initial smoke test, the software is handed back to the development team, where it is fixed and then sent back to QA. Another set of smoke tests is performed on the software, and if it passes, the software is integrated with an existing build in the QA and staging environment. The software is now ready for more rigorous functional testing.

This cycle helps identify basic and critical issues in an application before the team working on it performs more in-depth testing. This is why smoke testing is performed at the initial phase of the testing lifecycle.

Smoke testing examples

Smoke testing tests the basic functionalities and core features of software in development. An example of a smoke test could be the testing of a login function of a web application. Performed from the perspective of the user, the page would be launched, and the tester would enter a valid username and password and attempt to sign in. A successful test would result in a successful login. If the feature doesn't work, it is sent back to the development team.

Another example of a smoke test could be the testing of if an item can be added to a cart in an e-shop. Again, a tester would launch a specified page and attempt to add the item to their cart. A successful test would see the item added to the cart, while a failure would result in an item not getting added to the cart.

Another feature, such as the search function in an application, could also be smoke-tested. The application would be opened up, and the user would then navigate to the search bar and search for and select a file to see if they can access it.