How Does Behavior-Driven Development (BDD) Work In an Agile Setting? (Part 3)

In the second post of this series, I explained why—contrary to the way that BDD has been popularized—it is not usually feasible for a Product Owner in a large organization to maintain BDD test specs, and what you should do instead. Next I want to talk about what kinds of testing are well supported by BDD tools.

Behavioral Tests Are Integration Tests

It is important to realize that BDD tests are inherently integration tests! A BDD test is a functional test, and it tests how the application will behave as a whole. It does not test a component. To test the application as a whole, you need to build the whole application. You might even want to deploy it the way that you would deploy for production, or you could simply run tests against the built artifacts. Let me explain what I mean.

BDD tests are inherently integration tests.

Local Versus Orchestrated Testing

Programmers normally have very convenient tools for building and testing their application changes. For example, there is a whole ecosystem of tooling around the Eclipse tool and other Java-centric tools, as well as Microsoft’s Visual Studio. Thus, for example, one can make changes to Web application code and with a single click deploy the changes and run the Web application locally on one’s laptop. That is a very rapid turnaround process, and it helps programmers to quickly write new code and debug it. Programmers can run tests directly from such an environment, and most of these tools now support BDD style tools. For example, I recently implemented a suite of BDD tests using Visual Studio and the SpecFlow testing tool, which supports gherkin. SpecFlow is a plugin for Visual Studio, and so I could run the tests directly from Visual Studio.

Unfortunately, running tests directly from a development tool, while powerful for finding most errors, does not find many errors that pertain to how the application runs in an actual production configuration, or how it interacts with other applications or components such as Web services, production databases, and so on. To test those things you have to do an actual deployment, including all components, to an isolated test environment. Therefore, it is important to have a progressively more realistic and complete set of tests that are run automatically at regular intervals. These are the “orchestrated” tests I referred to above. They are usually called “continuous integration tests”, but that term no longer makes sense, since today local tests can also perform some integration testing. Today’s “shift left” testing therefore looks something like what is shown in the figure, which is quite different from how things were done in early Agile teams, which focused on automated unit tests but did most other things manually. (See this article for a comparison of traditional Agile and newer DevOps practices.)

Can BDD Tools Be Used For Other Kinds of Tests?

BDD tools are useful for most other kinds of tests as well. You can even use them for performance testing! (To do that, just call a performance framework like Gatling from the test implementation code.) Some of the common types of tests that can be very conveniently run and managed with BDD tools are,

1. User level functional tests - how simulated user actions cause the application’s components work together to provide the functionality defined by user stories
2. API level functional tests - how the business logic of the application behaves.
3. User scenario, aka “use case level” tests - tests that verify that users can accomplish entire tasks that might span multiple application features
4. Error path tests - Test all the “edge cases” and how the system behaves when all the things that can go wrong actually do. It is often best to include these alongside the “happy path” tests.
5. Security story tests - Tests that verify that security features work, and test that simulate users attempting to harm or corrupt the application.
6. Larger scale integration tests - how the application works with other applications.
7. System component failure tests - verify that the application is resilient when components fail.
8. Performance tests - Verify that the application can handle load spikes, running for long periods, and maintain responsiveness during normal and typical peak loads. Also verify that high concurrent load does not result in data corruption or other anomalies.

Why would you use BDD tools for all these things? The big advantage of BDD tools is that they decouple the test spec from the test implementation. That is very powerful. The test spec actually drives the test, finding the test implementation code along the way. This means that when running tests you are thinking in terms of the specs—the desired behavior—instead of in terms of test modules. For example, your performance test spec can specify the performance SLAs, and then the implementation can verify those.

In the next and fourth post in this series I will explain the immensely important concept of API level tests, why most of your tests should be API tests, and why BDD tools are great for that.