Why code coverage?

In my previous article, I wrote about what is testing and why we should test, code coverage is the next level after applying good practices about testing.

So the answer to the question Why code coverage? is simple:

Because writing tests is easy.

Naturally, the following questions arise:

1. Am I testing my application enough?
2. Do I have enough tests?
3. Am I over-testing?
4. Do I have redundant tests that can be added to the build time?

We have all those questions going on... here is where test coverage comes in.

Example "to-do" app

Let’s say we have a simple "to-do" application:

Feature A: “User can add items”

Feature B: “User can complete todo items”

Feature C: “User can delete todo items”

Then we start to write tests:

How do you know you cover all the test case scenarios?

One option requires a really good up-to-date test case management system.

• You write the first test and add the case to the management system.
• You write the second test and add the case to the management system.
• And so on...
• Then you have hundreds of tests! it could look something like the following:

Keeping track via code coverage

To not be in the above situation we can indirectly measure all the features we are testing via the following 3 steps:

1. Instrument code
2. Run tests
3. Report results

To figure out instrumentation we can run the following little program:

If you run it you should see this result:

Which statements did not execute?

Line 5 from our little program did not execute meaning that the letter "c" was not printed out.

How to track it? how to prove it?

We could create an array of counters for example:

If we execute this tiny program we should get:

And therefore we could have the following report:

If we go even further we can get a nice HTML report:

The above is all cool, but we don't have to do this by ourselves, we use a library:

After we installed nyc we can remove the counters implementation from our previous example:

We can name this program instrument.js and we can run the following:

Then we should see something like the following:

And if we want to see a nice html report we can run the following:

And we should see a nice overview html report:

If we click instrument.js then we see a nice line by line html report:

Code coverage process

In the real world we could use jest or cypress to generate those reports for us on an automated way.

So the process is like the following:

1. Instrument app code.
2. Run Jest / Cypress tests.
3. Save collected code coverage information.

How to instrument your application code?

• Cypress has some guides and plugins like cypress/instrument-cra
• Jest has a coverage report provided out of the box (babel or v8)

100% = 0 Bugs?

Unfortunately, no.

One reason could be that we have unrealistic tests or subsets of inputs. Another reason can be that the code does not implement the feature correctly.

Naively aiming at 100% code coverage is not a good idea.

However, if we allow excluding code that doesn’t need to be tested from the coverage metric, aiming at 100% becomes much more meaningful and it becomes easier to keep a high test coverage due to psychological effects.

It is important to keep the focus on implementing the feature itself and not try to achieve 100% code coverage stubbornly since we are risking having false positives in our testing process. I think if we focus on implementing the feature correctly and have good test case scenarios the code coverage will naturally fall in place.

Trying to achieve a high coverage percentage might depend on at what stage the project is. For example, if the project is in the early stages, setting a high number might not be a good idea because features can change more often than not.

What’s your take on 100% code coverage?

Summary

In this article, we answered questions like Am I testing my application enough? or Do I have redundant tests that can be added to the build time? by going deep into how Code Coverage tools work. It is important to understand those tools and that they do a good job of helping us to test our application to keep them robust and reliable however we should remember to stay focused on the feature at hand.

Thanks for reading.

Marcelo.

#codecoverage #automatedtests #softwaredevelopment