Tests should do what they say and say what they do - do yours?

Assertions are the beating heart of your automated tests - but are yours doing their job properly?

The role of an assertion is two-fold: it should tell you what the expected outcome of the test is (and not make you have to work too hard to figure it out), and also help you troubleshoot if something goes wrong.

But many assertions in real-world code fail on one or both of these points.

Here are a few common anti-patterns I see when it comes to assertions (I'll use Java and AssertJ as an example, but similar patterns exist in tests and test frameworks for any language):

Avoid using assertTrue() or assertThat(...).isTrue().?

Suppose we have a list like the following:

var colors = Arrays.asList("RED", "GREEN", "BLUE");        

If we want to check that this list contains the color yellow, we could write the following assertion:

assertThat(colors.contains("YELLOW")).isTrue();        

But while this isn't wrong per se, it obscures the nature of what's being tested and makes the failure message less informative. If the test fails, it will only tell you that a boolean was expected to be true but was false, without providing any context about the actual list colors.

A much better approach would be to use AssertJ as intended, like this:

assertThat(colors).contains("YELLOW");        

This is both readable and effective: if this test fails, the error message will be much more informative, telling you exactly what was expected (that "YELLOW" should have been in colors) and what was actually observed (the actual contents of colors).

Make sure your assertions assert something

I recently saw some code like the following:

assertThat(colors.size() > 0)        

At first glance this might look OK, but it is not actually an assertion at all. It's just an expression that doesn't assert anything. The condition (colors.size() > 0) is evaluated, but its result isn't used in any way. If colors is indeed empty, this line will not cause a test failure as it should, because it lacks the concluding .isTrue() that would turn it into an actual assertion, resulting in a test that will pass even if the condition isn't met.

Once again, the correct approach is to use the idiomatic AssertJ assertions:

assertThat(colors).isNotEmpty();        

This version directly asserts that the list colors is not empty. It is more expressive and clear in its intent, and will provide a more helpful error message if the test fails.

Make sure what you assert is meaningful

Another common antipattern I see is where a test doesn't check anything meaningful.

List<Product> products = productService.getProducts();

assertThat(products).isNotNull();        

While this is a valid check, it's often not sufficient for most testing scenarios. The test would pass even if the service returned an empty list, or a list with incorrect or unexpected products.?

A better approach would generally be to test that the returned data contains the expected values. For example, if we wanted to check that specific products were returned, and that the price values were all defined, we could write more expressive assertions like these ones:

assertThat(products).extracting(Product::getName)

????.containsExactlyInAnyOrder("Product1", "Product2", "Product3");

assertThat(products).extracting(Product::getPrice)

????.allMatch(price -> price > 0);        

In conclusion, crafting meaningful assertions is a crucial aspect of writing effective automated tests. The difference between a good assertion and a bad one can influence how quickly and efficiently a problem in the code is detected and fixed. It can be tempting to write quick, simple assertions that just check for non-nullity or use basic equality, but this can often lead to tests that are less helpful than they could be.

By understanding and avoiding common assertion anti-patterns, like the ones discussed in this post, developers can ensure their tests are clear, robust, and provide valuable feedback when things go wrong. Using specific and descriptive assertion methods provided by libraries like AssertJ helps in achieving this. Remember, an assertion should always make its expectations explicit and should provide as much context as possible when those expectations aren't met. Happy testing!