Unit Tests: Good or Bad? (part 3)

Part three of three: Use the Force!

In the first two parts, The Mind Set and The Dark Side, I have demonstrated that unit tests can be either good or bad and I have provided a non exhaustive list of practices that lead to bad unit testing. In this last section, I will attempt to put forward what should be the correct way to achieve good unit testing.

So yes, there is a right way... but there is no silver bullet. Indeed, writing useful tests will require that you stretch your design muscle and in particular that you apply most of the so called SOLID principles [12] in a repeated fashion.

General qualities of good unit test should be:

• Tests should be written before the code
• Test the outcome, not the process
• Use the right tool for the job

Tests should be written before the code

This is the essence of TDD [9] and probably the most difficult part to master. Not recognizing this simple fact can lead to the impression that the tests can be implemented as an afterthought which is obviously not the case.

The Three Laws

Uncle Bob's Three Rules of TDD: [4,5]

• Write no production code except to pass a failing test.
• Write only sufficient tests to fail.
• Write only sufficient code to pass the failing test.

In theory, this approach looks pretty simple and indeed it is for trivial projects. At its core, it supposes that you start the coding without even knowing where you are going. Design is no more a pre-construction of the solution to implement, but instead a work in progress toward the goal to achieve.

Where to start?

At first the problem can be seen as a double design challenge: design the tests and the code in parallel. But this should not be the case since we should add only one test at a time.

So which test should we write first? If too many ideas come to your mind, maybe you are trying to do too many things at once. Is it possible to split the problem into sub issues that can be addressed separately?

“If a test is painful to write, make the System Under Test easier to use.” -- Mark Seemann

The first tests should be directed toward the main API [6]. This of course can be seen as a top down (aka outside-in) approach. At this point, you still have no idea how the problem will be solved and this is ok since you can still expect the API to change over development time.

Reciprocally, the first code will only need to implement these first tests. Some part of this new code will probably need to be mocked. This may sound weird at first, mocking the production code, but you have probably already done it in the past without even thinking about it.

Deferring

Adding more tests to the project will provide more information about the choices that have to be made at the implementation level. This is directly in line with the controversial “Architecture Decisions Should Be Made At the Last Responsible Moment” aka “Real Options” [7] based on the Poppendieck’s Lean Software Development doctrine [11].

As you go down toward the solution of the problem, you will need to access some external interfaces that can behave in some very special and unexpected ways. So what happens when your wonderful design just doesn’t fit anymore? The good news is that you now have all the unit tests that you need to redesign your solution. This is indeed the ultimate way to assess your unit test design robustness!

“Tests need to be designed. Principles of design apply to tests just as much as they apply to regular code.” -- Uncle Bob

Test the outcome, not the mechanism

First of all, your tests should be functional (think input vs output). The main idea is to avoid mirroring the process of the System Under Test (SUT) inside the testing methods which would forcibly lead to a fail test as soon as the system is updated. This is also in line with the previous approach of writing your tests against the main API [6], so it should not come as a surprise.

At this point, one may argue that this is no more Unit Testing but rather Integration Testing. Indeed, this is right, but TDD is no synonym of unit testing either. Most bugs don't happen inside, but between the objects [8]. But, what’s the “Unit” anyway?

My epiphany moment…

Simply put, I realized that the main misconception I had came from the concept of “Unit” itself. Indeed, a unit should not be associated directly to a class, but instead, it should be associated with the new feature being implemented. If the new feature requires a group of strongly coupled classes, then these classes should be grouped into a package [12] and this package should be considered as the unit to be tested. That’s it!

“Testing still must be done, and should be done, at the system level: where the value is.” -- James O Coplien

Low coupling, low mocking

This is the main challenge. As previously mentioned in my friend's paper [3], low coupling can be obtained by means of the Dependency injection principle (the “D” in SOLID [12]) at the price of creating a new interface. But this is acceptable since we only need to do it at the package API level, n'est-ce pas?

A very interesting example of low coupling can also be found in this JMockit tutorial [13]. It points to a fully tested re-implementation of Spring’s PetClinic which does not contain any fake objects! The tests are written against the API of the higher-level components of the application.

• The tested object’s parameters are automatically generated from the existing concrete classes, so there is no need to create numerous interfaces.
• The database is emulated by using an in memory HSQL instance.
• There is no explicit mocking done, whatsoever!

Creating a mock against an interface should limit the need to change the tests when the code or its abstraction is changed. But in its “Mocking Mocking and Testing Outcomes” [14], Uncle Bob prefers to give us an example of how to Mock an object without actually using Mocks but in some other twisted ways…

“Mocks, by their very nature, are coupled to mechanisms instead of outcomes.” -- Uncle Bob

Boundaries

Unit tests should treat the classes they test as black boxes. But what happens when the black box requires some external objects to work? This is the so-called boundary case [15,16]. 

Indeed, the use of mocks can be associated with a smell, but they may be required when the function needs to access some external resources or modules on which we do not have any control. And this is exactly where they should be used!

How? One should use the humble principle [2] and split the method to isolate the boundary part, i.e. the part that is calling the database. That method should then call the external system through a dependency injected mocked interface.

Do not write useless tests just to cover the boundary methods that cannot be tested. If any boundary methods happen to not be covered by the unit tests, my advice here would be to use some kind of reusable, browsable annotation instead, aka JaCoCo’s @ExcludeFromCodeCoverage

“Mock across architecturally significant boundaries, but not within those boundaries.” -- Uncle Bob

Attitude over tools

As previously stated, TDD is all about design. So your first good reflex should be to dig into good architecture and design books.

Be Humble

The Humble Object Pattern [2] will help to solve most of the problems that you are facing when trying to implement your unit tests. Simply stated: Hard to test objects should be decomposed in simpler objects that are easy to test. This obviously has a direct impact on the code design, but this will be for the better.

Be SOLID

SOLID principles [12] are your best friends when designing your code. These principles are all geared toward the dependency management which is at the heart of the TDD philosophy.

Be Test Driven

Putting all the previous together helps to remove the focus off the unit test and rather on what should be good TDD practice. For that matter, the best one can do is to precisely adapt its practice to the problem at hand.

Be warned

• Don’t get fooled by the coverage hype.
• Beware of the “helping” tools.
• A non-trivial unit is not a Class, but a Package.
• Don’t write useless tests.
• Don’t test the Mocks.

Conclusion

So are Unit Tests good or bad? 

Indeed, Unit Tests can be very handy... when done right!

But doing them right is quite an endeavor if someone wishes to attain the full benefits of TDD. “Good Testing”, instead of “Unit Testing”, should be what we strive for!

Luke: ... Is the dark side stronger?

Yoda: No, no, no. Quicker, easier, more seductive.

Luke: But how am I to know the good side from the bad?

Yoda: You will know... [10]

References

[1] Yoda image by Mario Eppinger, https://pixabay.com/en/yoda-star-wars-jedi-figure-toys-3888783, downloaded January 8th, 2019.

Other Star Wars pictures have been found on Google using “Labeled for Reuse” from tools.

[2] TDD Patterns: Humble Object by Ilija Eftimov, https://ieftimov.com/tdd-humble-object, consulted on January 22th, 2019.

[3] A Case for Unit Tests by Frederic Simard, https://www.dhirubhai.net/pulse/case-unit-tests-frederic-simard, consulted December 17th, 2018.

[4] The Three Laws Of TDD (video) by Robert C. Martin, https://youtu.be/qkblc5WRn-U, consulted November 20th, 2018.

[5] The Three Rules Of Tdd by Robert C. Martin, https://www.butunclebob.com/ArticleS.UncleBob.TheThreeRulesOfTdd, consulted January 9th, 2019.

[6] TDD Harms Architecture by Robert C. Martin, https://blog.cleancoder.com/uncle-bob/2017/03/03/TDD-Harms-Architecture.html, consulted December 20th, 2018.

[7] "Real Options" Underlie Agile Practices by Chris Matts & Olav Maassen, https://www.infoq.com/articles/real-options-enhance-agility, consulted January 9th, 2019.

[8] Why Most Unit Testing is Waste by James O Coplien, https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf, downloaded December 20th, 2018.

[9] Introduction to Test Driven Development (TDD) by Scott Ambler, https://agiledata.org/essays/tdd.html, consulted January 8th, 2019.

[10] Star Wars: Episode V - The Empire Strikes Back by George Lucas, https://www.quotes.net/mquote/91421, consulted on January 22nd, 2019.

[11] Poppendieck, Mary B., and Thomas David Poppendieck. 2013. Lean software development: an agile toolkit. Boston [etc.]: Addison Wesley.

[12] The Principles of OOD by Robert C. Martin, https://butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod, consulted January 9th, 2019.

[13] JMockit tutorial, https://jmockit.github.io/tutorial/EnterpriseApplications.html, consulted January 7th, 2019.

[14] Uncle Bob about Mocking Mocking, by Robert C. Martin, https://sites.google.com/site/unclebobconsultingllc/blogs-by-robert-martin/mocking-mocking-and-testing-outcomes, consulted December 17th, 2018.

[15] When to Mock by Robert C. Martin, https://blog.cleancoder.com/uncle-bob/2014/05/10/WhenToMock.html, consulted December 17th, 2018.

[16] The two places where TDD is impractical or inappropriate by Robert C. Martin, https://blog.cleancoder.com/uncle-bob/2014/04/30/When-tdd-does-not-work.html, consulted December 17th, 2018.