Pact Consumer-driven Contract Assurance: Beyond the promise of API Backward Compatibility

One of the key aspects of API first design based microservices architecture is the communication and maintenance of service contracts. These service contracts are usually the inputs and outputs provided by your API, or if you are implementing a REST based API, it’s a format of the JSON/XML or if its Queue message consuming service, this is the format of the message on the queue.

In all scenarios your microservice contracts should be visible to consumers of your services, developers of your services and the product owners who own your services as part of their API catalogue.

When we introduce new improvements with additional service and updates to existing.. how do we check contract is still valid. What assurance is that you do not un-know break the contract and bring the chaos situation

So let us understand the ground facts here..

First of its a Multi Player Game. For every API, here are four main groups of stakeholders: API providers, API customers, API consumers, and end-users.

API providers build, expose, and operate APIs. API customers make a decision to buy the use of an API. API consumers develop apps that use APIs. End-users don’t use directly APIs. Instead, they use APIs indirectly via the app that is developed by the API consumer and provided by the API customer.

But how we can ensure all players are well playing..? Do we have any report to assure that?

Let's begin by looking at the state of Micro service testing today

Does it provide any assurances? The Test Pyramid. This concept was developed by Mike Cohn at a time when most software testing was done using manipulation of the software’s interface

1. Unit Tests and Integration tests validate a system on a development machine pretty easily, and only consume resources when testing.
2. End 2 End integration tests require a large external staging environment that is difficult to set up, must be shared between developers, consumes resources 24x7, and requires external access.

Let us dig bit deeper into Integration Tests

The aim is to verify the external behavior of a single service. The test framework starts an instance of the service, and then uses the service’s external interface to execute the business logic of the service in the same way that an external service would integrate with the tested service. For example, a REST API service would be tested by making HTTP requests to the service.

In this example, we have a provider “Person” service which expose an API that given an id, it replies an Person JSON response. This API is consumed by another service.

To summarize:

endpoint : GET /api/persons/{id}

response body : {"id": "1", "name": "Joseph”}

To test this, what can we do? mock?

Mocks are a type of TestDouble .. Test Double? A test double is an object that can stand in for a real object in a test, similar to how a stunt double stands in for an actor in a movie

Test Double define a sort of specification based on expectations and, in this particular case, mocks can substitute APIs or clients reducing in this way, parties to set up and run.

Above is an example, but there are many alterations solutions to load part of an application which doesn’t imply a real execution. however, you can simulate an interaction through a mock or stub based on Contracts.

As showed before, we can have two sides of breaking change based on the side that doesn’t maintain expectations and in the same way, we can identify two sides of contracts:

• Provider Contracts
• Consumer Contracts

What if we have a solution that you can win both worlds? What I mean is:

• All the benefits of using mocks — quick tests, easy environment setup, and deterministic tests.
• All the benefits of a fully deployed E2E — reliability and confidence when deploying to production.

WIN-WIN Solution : Contract Driven Testing

To put it simply — with contract testing, you work with mocks that represent the agreed-upon API contract between the services, but also, the services (or teams really) that provide the API for consumption have to uphold that API contract that is used in those mocks, and the contract is enforced in their CI so they can’t break the API and go to production.

Who’s who in Consumer Driven Contract Testing

1.    Consumer — any party that interacts with a dependent service through an API (HTTP, event-based, etc). This often drills down to be a backend interacting with another backend API, but it doesn’t have to be. A browser frontend is also a valid party that depends on a backend API and is considered a consumer.

2.    Provider — any party that provides a service for interacting with to its dependents.

3.    Contracts — just like law enforceable agreements, these contracts represent a set of interactions with expected request and response structures. Through-out this document we will use Pacts and Contracts interchangeably to refer to the contract.

4.    Broker — the contracts need a place to be stored. It can be any generic assets server, but its better if they are versioned, so version control is a choice.

For the broker it could be as shared network drive as long as its kept audited and change controlled. A better choice is to use the Pact Broker which is suited for this exact need and is open source

So now we know who are the players in the testing game.. Let us look at its lifecycle

Consumer Driven Contract Testing Lifecycle

Following is a high level overview for the workflow of contract testing between a consumer and a provider team.

The cycle starts with the top-left side of the picture where the consumer team initiates a collaboration about the requirements for the contract with the provider team.

Once both teams have agreed on the API contract, the consumer team can venture into writing tests that manifest the interactions and their expectations. The result of running these tests will become the contract, in essence, being a JSON representation of the API contract specifying all the expected interactions.

Pact enables consumer driven contract testing, providing a mock service and DSL for the consumer project, interaction playback and verification for the service provider project.

The Pact family of testing frameworks (Pact-JVM, Pact Ruby, Pact .NET, Pact Go, Pact.js, Pact Swift etc.) provide support for Consumer Driven Contract Testing between dependent systems where the integration is based on HTTP (or message queues for some of the implementations).

Let us experience step by step

Consumer side : We are driven by Consumer, so then we start working on consumer side

Step 1 : Add the pact maven dependency in consumer side

<dependency>
    <groupId>au.com.dius</groupId>
    <artifactId>pact-jvm-consumer-junit5</artifactId>
    <version>4.0.10</version>
</dependency>

Step 2: Examine consumer expectation

The provider is a person service which has the endpoint GET /api/persons/{id} which returns a user’s data in the following form:

{ "id": "1", "name": "Joseph}

 It expects that a call to GET /api/persons/{id} returns

• a 200 success code,
• content type JSON with UTF-8 encoding and
• a JSON body that contains the field name of type string

All the code can be found in this GitHub repository.

Now the easiest way to create the Pact file is via a unit test. The test goes in the same directory as all the other unit tests of the consumer. The unit test will do two things:

1.    verifies that our code can handle the expected provider responses

2.    it creates the Pact file.

But wonder which code should be tested in order to verify that the expected provider response can be handled? A good starting point is the class that directly interacts with the provider. In our example, we have a PersonServiceClient that provides a getPerson method. This method calls the user service via a RestTemplate, parses the response into a Person DTO object and returns it:

We need to write unit test that will perform the following steps:

1. Start a server that mocks the provider with the given interactions.
2. Call the getPerson method which will call the mocked provider.
3. Assert the returned PersonDTO object.
4. Write the Pact file based on the given interactions.

The interactions will be defined in a separate method, annotated with @Pact.

Here, Items 1. and 4. will be handled by the Pact framework’s Junit rule Items 2. and 3. will be in a regular @Test method.

Step 3 – Write the unit test

As a first step we create the unit test class and add the test method:

Step 4- define the interactions

Here, we will define the interactions by creating a method annotated with @Pact and the consumer name. The method returns the description of the contract using the pact-jvm Lambda DSL.

Matchers: build a response with PactDslJsonBody On the provider side, the test verify that the generated response is perfectly equal to the one defined in the contract.

Pact DSL

The Pact testing framework provides also a DSL that permits the definition of different matching case in this way:

Here we created a response with PactDslJsonBody DSL that defines a match case based on type instead of value. It’s possible with PactDslJsonBody different match case based on regex or array length.

The Pact DSL provides a fluent API very similar to Spring mockMvc: Here we are saying that when the mock server receives an Request, it should return 200 and an Response.

Couple of things to keep in mind for our understanding..

1.    uponReceiving(...) is just the description of the contract.

2.    given(...) can be used to prepare the provider i.e. bring it into a certain state

3.    It is very important to use stringType or numberType instead of *Value for example : stringType generates a matcher that just checks the type whereas stringValue puts the concrete value into the contract. It might be tempting to use the real user name since we know what it is. However, this again leads to a tight coupling between the consumer and provider. (If the name changes, the tests will fail.)

The last step is setting up the mock server. This is done by adding the PactProviderRuleMk2 rule to the class and annotating the test method with PactVerification and the name of the previous method. This annotation is important because it tells the Pact provider rule to start the mock server with the interaction defined in the given method (which is called fragment here for some reason).

Step 5- Verify the contract

Now we can create the real test which verify the contract on the consumer side:

Step 6- Generate Contract

run mvn test now and we will see in ./target/pacts a json file that use the pact formalism for contracts. We use the generated contract in the provider-side.

Step 7 Broker Setup with docker-compose

Broker — the contracts need a place to be stored. It can be any generic assets server, but its better if they are versioned, so version control is a choice. A better choice is to use the Pact Broker which is suited for this exact need and is open source, and

Broker Setup with docker-compose

To get a running Pact-Broker via docker-compose we put a file called docker-compose.yml in the root of our project.

Run docker-compose up

Step 8 – Publishing to Pact Broker

To upload the Contract add the following plugin to the consumers pom.xml

<plugin>

        <groupId>au.com.dius</groupId>

        <artifactId>pact-jvm-provider-maven_2.12</artifactId>

        <version>3.5.11</version>

        <configuration>

                <pactBrokerUrl>https://localhost:80</pactBrokerUrl>

                <projectVersion>${project.version}</projectVersion>

                <trimSnapshot>true</trimSnapshot>

        </configuration>

</plugin>

Afterwards you are able to upload your contract to the broker by executing the following command:

mvn verify pact:publish

Verify the pack broker

Provider: Verifying the contracts

We will now show how to create the tests on the provider site that verify that the contracts are fulfilled. We use Spring Boot integration tests for this because it allows us to

• easily mock away any downstream systems we don’t want to depend on e.g. a database and
• to run the tests as part of our normal build.

For the test we create a regular Spring Boot web integration test and use the SpringRestPactRunner Junit runner.

Additionally, we need to add the following annotations to the class:

• Which Pact files to load by specifying the @Provider annotation with the provider name.
• Where to load the Pact files from by specifying one of @PactBroker or @PactFolder
• The target: Where to run the interactions against and verify the responses. The SpringBootHttpTarget is for the Spring Boot integration tests. The tests are executed against the application started by the integration test on the random port. There are other targets e.g. MockMvcTarget which we have successfully used in a plain spring application where we run the test with just the controller.
• A method for each provider state given in the contract. The method can be used to set up the desired provider state e.g. creating the user in the database or mocking the service provides the user.

Here is the thing: we need to verify the contract against provider implementation. In the Spring world, it’s sounds like an integration test which verify the web layer. So here the magic:

That’s it. As you can see, we have a @SpringBootTest with random ports and so on, but the main thing here is to bind both context together.

The @State annotation : the given statement in the consumer contract, define with a business expression, the state in which the system-under-test, should be during the execution. Following this approach, we define in the provider, a method with @state annotation, that contains the commands necessary for the correct execution.

In our case, we mock the business service delegated to execute the eco logic. The framework executes the state method before calling the API defined in the contracts. The real test, in this way, is “reduced” to a simple call:

    @TestTemplate
    @ExtendWith(PactVrificationInvocationContextProvider.class)
    void pactVerificationTestTemplate(PactVerificationContext context) {
        context.verifyInteraction();
    }

So what does it bring to our game plan?

1. Clear Well Established Consumer driven process — if the contract is consumer driven then the interactions that are built are actually per an expectation, instead of being generic endpoints and interactions that no one uses.
2. Win both Confidence and Speed — If you bring up a full E2E environment with dependencies that’s going to be reliable, but very complicated and slow. If you use mocks, you get the other half of it — easy and fast, but not so reliable.
3. True release independence — Services can be deployed independently, not requiring a full system, with high confidence for API contracts being uphold.
4. Insight into API consumers — have you ever wondered “which consumer is actually using this order field ? is it still swift api version x being used?” so now you know. Using the Pact broker you get insight into all consumer and provider connections as well as the specific usage of an API response per consumer.

So why Pact and not other CDC like spring-cloud-contract?

Spring-cloud-contract is a similar framework and, from an implementation side of view, not very different from pact. But, then, why do we choose Pact?

Because Pact is usable from many languages aside from the JVM ecosystem: There are Javascript, Python, Ruby, Go, .Net and other implementations. Typically you have your microservice infrastructure (which may or may not all be JVM based) and one or more frontends talking to the microservices. So the contracts can be made from any of these…

so in a microservice landscape, the advantage is huge.

Wrap up

We learned in this how to set up consumer-driven contract testing with Pact. The big advantage over other kinds of tests is, that it breaks your build when you break the contract with the consumers on the provider side.

Consumer-driven contract testing will prevent the APIs you provide and consume from breaking unexpectedly. This fosters further development and introducing (breaking!) features!