Testing an API without E2E tests?! (Node + Express + Supertest example)

“I haven’t seen it working yet!”

Many test automation engineers (and developers) would say that when discussing the test approach for an API. “I haven’t seen it working yet, we need end-to-end tests!” They would say that and make some sarcastic “unit tests were passing” remarks (and post some memes sometimes! ??)

Are they wrong? ??

Or, perhaps, they are right and you need to always rely on some form of the end-to-end? Or truth is somewhere in the middle?!

To the rescue! This article will:

• help you UNDERSTAND an alternative way of testing your API
• demonstrate a PRACTICAL EXAMPLE of such testing using Node.js, Express.js, and Supertest library
• and highlight the PROS & CONS of this approach compared to the “traditional end-to-end” testing

What is end-to-end?

To avoid confusion, let us start with discussing the meaning of an “E2E (or end-to-end) test”!

End-to-end test executes all layers of software from one end (aka ENTRY point) to another end (aka DEEPEST dependency, like 3rd party system, database, or another API under your control)

Look at this picture:

By this definition, BOTH YELLOW AND GREEN highlighted areas are E2E.

The green one is an API E2E test. The yellow one is a UI E2E test. Both are end-to-end (fight me in the comments if you disagree ??). In addition, the yellow one is also a System test as it tests the WHOLE THING.

API inner structure

Now look at this (simplified) model explaining what is happening inside of the typical RESTful API when an HTTP request touches it:

This picture is a generalisation, of course, there might be slight differences in your API. Request enters the API and goes through the:

1. middleware (aka ”common tasks” for all requests like checking authorization/authentication or logging details of this request)
2. routing (find an endpoint that will be doing the work)
3. controller (a thing that “controls” what work needs to be done)
4. business logic or database layer (aka ”the work” done by your API)
5. (omitted from picture) middleware (usual examples are output formatters and exception handlers)

…and exits your API in the form of a Response! ????

Traditional API testing approach

Usually, APIs are tested by using an HTTP client. Sometimes this client has a UI, like Postman, other times it’s just a library allowing you to send HTTP requests (like Axios/Got/Fetch running in the test framework of your choice).

What is important is that normally THESE TESTS have an EXTERNAL PERSPECTIVE at the API under test. They are running outside of it, as a separate program. The interaction with the API happens via HTTP calls.

Request → Response!

The benefits of this approach are:

• ? Realistic. You are consuming this API in the same way the clients (like your UI) would.
• ? Validating the deployment process, test data setup, and infrastructure state as a side effect.

Downsides are:

• ? Lack of control. Testing only happy cases would work great, but, for example, how will you test the response behaviour when “something crashed” within the API?
• ? Mocking dependencies is (somewhat) painful. If you have a 3rd party API you talk to, you could deploy a “standalone mock server” to act like this API. Same with the database. Possible, but it creates complexity! This “mock server” means extra code, extra configuration, extra infra, extra deployment pipeline, a potential “forgetting something” risk, etc.

Magic (aka the solution)

To overcome the downsides of the “EXTERNAL” tests, you can create some tests that are "INTERNAL" to your API.

What does it give you? You probably guessed already!

CONTROL! ??

But it is easy to say “You can create it!” The question is how? Good thing that I have an example for you:)

How to do it

Please note, that despite provided code examples below, you might not be able to reproduce it by simply typing this code in. Some parts of it (like Jest/Babel configuration) were omitted for simplicity and more concise and pleasing visuals. Goal of this article is to show that “this is possible” (aka “guidance”) not to teach you “this is exactly how you do it” (aka “recipe”)! ??

Let’s look at the code. Here’s a pretty standard Express API.

If you use the Supertest npm library, you can easily create a test for your API! It might look like so:

This is already cool! This code mimics the “external test happy case 200 OK” with one subtle difference! Tests start the API!

YOU HAVE CONTROL!

You have it because you start the API from WITHIN THE TEST! This line does the import and starts the API:

import app from '../index.js'        

And then the test invokes the app!

const response = await request(app).get('/items')        

So now the API is started from within the “test codebase”, so you can USE THE MOCKING capabilities of your test framework.

In my example, I use Jest as a framework of choice. Look at this example:

In the code above, the call to the db.getItems() function that fetches data from DB is mocked. Then the request goes through all of the inner API parts: middleware → routes to /items → controller → mocked db.getItems() call → and asserts on the mocked response!

Pretty useful, huh? And flexible!

You could say that this test is twofold:

It is an “E2E-test-like” (as in “testing multiple layers from one end to another”) and at the same time a “Unit-test-like” (as in “testing individual endpoint” and “having control over the runtime”).

And because of it, you get the best of both worlds! Schematically the things done by this test can be visualised like this:

But your “fake data” or “fake behaviour” could be applied to any part of the API now! Amazing? Amazing!

An attentive to details reader would probably ask: “why is the blue test box sitting inside the purple API box?” And you are right! Technically, API has been started from within the test, so the blue box should encircle the purple one. This will be “technically correct” but will lose some clarity for visuals of the “external & internal tests”. And I have decided that CLEAR VISUALISATIONS are more important in this case! ??

So, can you ditch the E2E?

…NO! ??

Having some form of E2E tests is crucial to understanding if your API functions within a given environment!

You need to utilize both approaches!

• ? EXTERNAL tests will give you confidence that IT WORKS (happy case tests in a given infrastructure/environment)
• ? INTERNAL tests will give you confidence that IT WORKS CORRECTLY (tests to validate the behaviour in specific scenarios with the help of “some magic” (aka “mocking”))

Note that the external tests do not necessarily need to be “tests tests” in a traditional sense. They are just checks! This can be achieved by your monitoring tooling (like “New Relic synthetic monitoring”) or even the infrastructure checks (like “load balancer health checks”).

The choice is yours!

The end

If you enjoyed the article don’t forget to react to it or leave a comment. ????

Also a reminder that I coach people on test automation within the JavaScript ecosystem. You can book a free first consultation to talk about your needs and goals here: https://ivanandcode.com/coaching

Follow me on Telegram to never miss a new content: https://t.me/ivanAndCode_channel