Ep.12: Functional Testing with Postman + Newman

This article is part of an in-depth comparison series of the top Node frameworks. We'll cover all the important aspects: market share, learning curve, ecosystem, security and more. To compare them properly, we will build the exact same app in all these frameworks (plus Vanilla Node), observe all the steps along the way and then benchmark them as we progressively add more functionalities.

Table of contents

1. Market Share Distribution Analysis: Picking the most used frameworks
2. Planning Phase: Use cases, Minimum Viable Product requirements, Architecture
3. Tools & Setup: Getting started and setting up the tools and environment
4. Scaffolding: Project Settings and Configuration, Common template repo
5. Express: basic app (MVP)
6. Koa: basic app (MVP)
7. Express Flavors: basic app (MVP) in multiple variants: OOP, FP, TypeScript, single-file etc
8. Nest.js: basic app (MVP)
9. Fastify: basic app (MVP)
10. Next.js: basic app (MVP)
11. Vanilla Node: basic app (MVP)
12. Functional testing with Postman + Newman

At last, we have the same app built in all the Big 5 Node.js Backend Framework, plus Vanilla Node! Now, how do we know they all work, and in the exact same manner nonetheless? We can either use a browser to manually navigate the 13 routes we've exposed, or we could use an HTTP client. There are several out there, Postman, Selenium, Cucumber, Insomnia, Thunderclient, Hopscotch and more. I ran a comparison last year and by far the most versatile and popular is Postman, so we'll go with this one. I did not have time to publish that, but if you want to have a quick comparison, I recommend Alex Xu's infographic.

Postman is free to use for teams of up to three users, which fits very well the team of ...me that I'm running here. It also allows you to do a limited number of collection runs each day. Since I'll run lots and lots of them, that doesn't suit my use case, so I'll add Newman to the mix, which is but a simple CLI for automating tasks with Postman. I’m going to use the Postman's extension for VSCode, to keep everything in one place, but you can choose your own. You'll need to create an account on postman.com to sync your collections and make the extension work.

So, let's go ahead and install them:

# on Windows
winget install -e --id Postman.Postman
npm install --global newman
code --install-extension postman.postman-for-vscode


# on a Mac
brew install --cask postman
npm install --global newman
code --install-extension postman.postman-for-vscode


# on Linux
wget https://dl.pstmn.io/download/latest/linux64 -O postman.tar.gz
sudo tar -xzf postman.tar.gz -C /opt
sudo ln -s /opt/Postman/Postman /usr/local/bin/postman
npm install --global newman
code --install-extension postman.postman-for-vscode        

Environment, Collection, HTTP Requests and Tests

Once we have that up and running, we want to create 4 things:

1. An environment - this is just a place where we store environment variables, like the port number (3000) and the root URL (HTTP://localhost:{{port}}). We can then access these variables with moustache notation, like so: {{port}} or {{root}}. Just click on Environments, add these two variables and save
2. A collection - a place where we will group our requests. We will need this so we can run all our requests at once, as there will be dozens or hundreds as we progress. Similarly, click Collections and make a new one.
3. A bunch of HTTP requests - i.e. calls to our route endpoints.
4. Functional tests with Chai - a way to check whether the calls were successful, whether the response was the one we expected etc.

Here's how we do the requests and tests (steps 3):

• Start any app with ‘npm run start’ in the terminal;
• Add ‘New HTTP Request’ (the biggest button, can’t miss it);
• Let’s start with a create request: select POST from the dropdown and add the route next to it: {{root}}/users/create;

Let's add a body to that request:

• Select the Body tab --> raw subtab--> JSON at the drop-down (it’s Text by default);
• Add a payload, like { “username”: “user4”, “password”: “pass4” etc... };

Testing is done using the Chai assertion library, and there's a handy code generator to the right. For our example, we want to check that the status code is 201 and that the response body contains a username "user" and a fullname "Gillian Beck". To assert that, just open the Tests tab and add the test below:

With that in mind, you can now create and test the GET, PUT and DEL routes as well.

GET and DEL are a bit different. You don’t need a Body payload at all, you do need to pass a parameter though, the user id (that’s why we have the?:id in the path?—?https://localhost:3000/user/:id). To do so, Click on Params (Where the green dot is) and edit the value for id in the Path variables table (I put 3 in the example below):

If you wonder what the Query Params are all about, they are for the situation where we would pass parameters in the Query, like so: https://localhost:3333/users/all?role=admin. In this hypothetical example, we would get all the users with the role of admin. But we have not implemented that yet.

PUT is similar to POST, it needs a Body payload and an?:id param. However, you don’t need both username and password in the payload, you can overwrite just one.

Test for failure

We have not implemented any error handling yet, so we expect a 404 at least if we try to access any inexistent route or method. Let's add at least one test for that:

Run manually

Well, let's put our requests to the test. Start your app with 'npm run dev', then click Send. You should see a response like this when you run the register user:

Great, it works! Now let's create the rest of the collection, or simply import it from the postman_collection.json in the repo.

Run Collection

Instead of manually running every request, let's run them all at once, like so:

It seems I've exhausted all my runs for today, but you should be able to do it. However, since I mentioned this 'exhausting' issue, is there any way to get around it other than registering for a paid account (which is totally worth the cost, especially in a larger team in a production environment)? Yes, there is: Newman to the rescue. Since we've already installed it, we can simply run it in the terminal:

newman run postman_collection.json --environment postman_environment.json        

I've added a script in package.json so that my lazy fingers won't have to type that hundreds of times, so we can run it also with:

npm run test:e2e        

The result is exactly what we hoped for, all our tests passed and the output is exactly what we hoped for:

Perfect! Now, since we have 6 repositories, some with multiple versions (about 20 different apps), we want to make sure that ALL of them pass these tests exactly the same way. This is where we start to appreciate not having to run 14 * 20 requests manually...

Fortunately, they all passed, and we can now say we have comparable apps with identical output and behaviour. Don't take my word for it, clone the repos and try them yourself.