The Pros and Cons of Using Pact for Contract Testing

In my previous blog post , I spoke about the importance of Contract Testing in the software development space. A tool that can assist in testing these service interactions certainly helps increase your ability to grow your contract testing coverage. And this is where Pact comes in.

Pact is an open-source contract testing tool used to test interactions between services based on contracts. It is particularly useful in microservices architectures where different services (or APIs) communicate with one another. Pact helps ensure that these services can interact correctly by testing the contracts between them.

As a reminder, here is a brief overview of the concept of Contract Testing and how Pact works in helping achieve these contract validations between services.

Key Concepts

1. Consumer: The service that consumes or calls another service.
2. Provider: The service that is being called.
3. Contract: A formalized agreement between a consumer and a provider, specifying the interactions and expectations, such as the request format, response format, and status codes.

How Pact Works

• Consumer-driven contracts: Pact allows the consumer of an API to define its expectations for the provider in a contract. The consumer specifies what it expects to receive based on its requests.
• Contracts as tests: Pact then generates a contract file (usually a JSON file) that contains these expectations. This file is used to verify that the provider can meet the consumer's expectations.
• Provider verification: The provider service runs tests using the contract to ensure that its implementation aligns with the consumer's expectations.

How Pact Helps with Contract Testing

1. Early Detection of Integration Issues: Pact helps detect potential mismatches between services early in the development cycle. Since the contract is created by the consumer and verified by the provider, both sides are aware of what to expect from each other.
2. Faster Feedback: Since Pact tests are focused on the interactions between services, developers can get quicker feedback on whether a change in one service breaks the contract with another.
3. Reduced Need for End-to-End Tests: By focusing on the contract between services, Pact reduces the reliance on slow, brittle end-to-end tests. Instead, interactions between services can be validated in isolation.
4. Supports Microservices: Pact is particularly useful in a microservices architecture, where services are developed and deployed independently. It ensures that service updates won't break their communication or integration with other services.
5. Can Be Integrated in CI/CD Pipelines: Pact contracts can be stored in a Pact Broker, allowing consumers and providers to continuously verify that they are compatible as part of a CI/CD pipeline. This makes it easier to manage and share contracts across different teams or services.

Example Workflow

1. Consumer Side: The consumer writes a test for its interaction with the provider, defining the expected request and the response it should receive. Pact generates a contract from this test.
2. Provider Side: The provider reads the contract and verifies that it can fulfill the expectations set by the consumer. If the provider changes in a way that violates the contract, the test will fail.
3. Continuous Validation: The contract can be shared using a Pact Broker, enabling continuous validation whenever the consumer or provider is updated.

How to Use Pact

It’s a concept that is easy to explain, but how you go about structuring your contract tests and making use of Pact to do so is key. Contract testing works best when integrating directing into your CI/CD pipelines so I also want to unpack how contract tests with Pact can be easily integrated into your delivery pipelines as well.

Below is an example of how to set up a Pact script for testing APIs and integrate it into a CI/CD pipeline.

1. Pact Example Script (Node.js/JavaScript)

We will start by setting up a script for both the consumer and the provider.

Pact Consumer Test (For Consumer-Driven Contract)

In this example, the consumer makes an HTTP request to the provider (an API), expecting a particular response.

Install dependencies:

In the below examples, I’ve included a quick bash script that makes use of npm can do this. Even if not using Node.JS though you can use similar commands to install the dependencies from other package management software.

npm install @pact-foundation/pact @pact-foundation/pact-node axios mocha --save-dev        

Consumer Pact Test (consumer.test.js):

All other non-install code makes use of JavaScript.

const path = require('path');
const { Pact } = require('@pact-foundation/pact');
const axios = require('axios');
const { expect } = require('chai');
?
const consumerPact = new Pact({
? consumer: 'UserService',??? // Name of the consumer
? provider: 'AuthService',??? // Name of the provider
? port: 1234,???????????????? // Port Pact will run the mock server on
? log: path.resolve(process.cwd(), 'logs', 'pact.log'), // Logging
? dir: path.resolve(process.cwd(), 'pacts'),?????????? // Directory to store Pact files
? logLevel: 'INFO',
});
?
// Define your expected interaction
describe('Pact with AuthService', () => {
? before(() =>
??? consumerPact.setup().then(() => {
????? consumerPact.addInteraction({
??????? uponReceiving: 'a request for a valid user',
??????? withRequest: {
????????? method: 'GET',
????????? path: '/user/1',
????????? headers: { 'Accept': 'application/json' },
??????? },
??????? willRespondWith: {
????????? status: 200,
????????? headers: { 'Content-Type': 'application/json' },
????????? body: { id: 1, name: 'John Doe', email: '[email protected]' },
??????? },
????? });
??? })
? );

? it('should receive a valid user', async () => {
??? const response = await axios.get('https://localhost:1234/user/1');
??? expect(response.status).to.eql(200);
??? expect(response.data).to.deep.include({
????? id: 1,
????? name: 'John Doe',
????? email: '[email protected]',
??? });
? });
?
? // Verify the contract
? after(() => consumerPact.verify().then(() => consumerPact.finalize()));
});        

This above consumer test does the following:

• Sets up a Pact mock server (consumerPact.setup()).
• Defines the expected request (/user/1) and the expected response (200 status with a specific body).
• Verifies that the consumer can successfully get the expected response.
• Generates a contract file (in the Pacts directory).

Pact Provider Test (Provider Verification)

This is simply the setup of the consumer. Now, let's verify the provider (API) against the contract generated by the consumer.

Install dependencies:

npm install @pact-foundation/pact-provider-verifier express --save-dev        

Provider Pact Verification (provider.test.js):

const { Verifier } = require('@pact-foundation/pact');
const path = require('path');
const app = require('./provider'); // Your actual provider API implementation
?
describe('Pact Verification', () => {
? it('should validate the expectations of UserService', () => {
??? return new Verifier({
????? providerBaseUrl: 'https://localhost:3000', // Where your provider is running
????? pactUrls: [path.resolve(process.cwd(), './pacts/userservice-authservice.json')], // Pact file generated by the consumer
????? provider: 'AuthService',
????? providerVersion: '1.0.0',
??? })
????? .verifyProvider()
????? .then(output => {
??????? console.log('Pact Verification Complete!');
??????? console.log(output);
????? });
? });
});        

• This script will verify the AuthService provider against the contract stored in the pacts/userservice-authservice.json file.
• The provider's actual implementation is expected to be running during this test (usually on port 3000 in this example).

2. Setting Up Pact in a CI/CD Pipeline

Now that we have a contract test written in Pact, we will look at how we can add this to your CI/CD pipelines using YAML.

To integrate Pact into a CI/CD pipeline, follow these steps:

1. Consumer Test in CI Pipeline

• Set up your CI system to run the consumer tests, which generate the Pact file.

Example GitHub Actions for Consumer:

name: Consumer Pact Test
?
on:
? push:
??? branches:
????? - main
?
jobs:
? test:
??? runs-on: ubuntu-latest
??? steps:
????? - uses: actions/checkout@v2
????? - name: Set up Node.js
??????? uses: actions/setup-node@v2
??????? with:
????????? node-version: '14'
????? - run: npm install
????? - run: npm test # Runs the consumer Pact test to generate the contract
????? - name: Upload Pact contract
??????? uses: pact-foundation/pact-action@v2
??????? with:
????????? pact-files: './pacts/*.json'? # Pact file to publish
????????? pact-broker-url: ${{ secrets.PACT_BROKER_URL }}? # Your Pact Broker URL
????????? pact-broker-token: ${{ secrets.PACT_BROKER_TOKEN }}? # Pact Broker token        

This workflow:

• Runs the consumer tests to generate the Pact contract.
• Publishes the Pact contract to a Pact Broker, where the provider will later validate it.

2. Provider Verification in CI Pipeline

This is only the first part of the pipeline setup though. After the Pact file is published, the provider needs to verify it.

Example GitHub Actions for Provider:

name: Provider Pact Verification

on:
? workflow_run:
??? workflows: ["Consumer Pact Test"]
??? types:
????? - completed
?
jobs:
? verify:
??? runs-on: ubuntu-latest
??? steps:
????? - uses: actions/checkout@v2
????? - name: Set up Node.js
??????? uses: actions/setup-node@v2
??????? with:
????????? node-version: '14'
????? - run: npm install
????? - name: Verify Pact contract
??????? run: npm test # Runs the provider Pact verification        

This workflow:

• Waits for the consumer test to finish (by using the workflow_run trigger).
• Runs the provider Pact verification to ensure that the provider matches the expectations defined in the contract.

3. Pact Broker

A Pact Broker is essential for sharing contracts between the consumer and provider. It allows continuous integration of Pact tests by:

• Storing and versioning contracts.
• Showing the status of the contract verification.
• Supporting environments (development, staging, production).

You can host your own Pact Broker or use a service like the Pactflow.io hosted broker.

Summary of Steps

1. Consumer generates a contract using Pact and publishes it to the Pact Broker.
2. Provider verifies the contract using Pact's verification tool, either locally or in a CI pipeline.
3. Integrate Pact into your CI/CD pipeline to ensure that each change is automatically tested, ensuring compatibility between services in a microservice architecture.

By setting this up, your API consumers and providers can evolve independently while maintaining reliable communication.

Pros and Cons of Pact

Now that we have a good idea of how to use Pact to perform our contract tests, let’s summarise the list of procs and cons we get out of the tool

Pros of Pact:

• Early Detection of Integration Issues: Pact enables the identification of mismatches between consumer and provider early in the development process, reducing the likelihood of integration failures during later stages.
• Consumer-Driven Contracts: Pact supports consumer-driven contracts, allowing consumers (clients) to define their expectations for the provider (API) and ensuring that providers fulfill those expectations.
• Microservice Scalability: Pact is ideal for microservices architectures, where different teams manage different services. It helps maintain consistency and reliability in communication between services, even when services are independently developed or updated.
• Reduced Reliance on End-to-End Tests: By focusing on the interactions between services, Pact reduces the need for complex and time-consuming end-to-end tests, leading to faster test execution and better test isolation.
• Version Control for Contracts: Pact Broker allows teams to store and share contracts across services, making it easier to manage contract versions and ensure that consumers and providers are always in sync.
• Clear Communication Between Teams: Pact makes the interactions between different teams more transparent. Each team knows the contract their service must adhere to, improving collaboration and avoiding misunderstandings.
• Test Automation & CI/CD Integration: Pact can be easily integrated into CI/CD pipelines, ensuring that contract validation occurs as part of automated testing. This ensures continuous validation of service compatibility after every code change or deployment.

Cons of Pact:

• Limited to HTTP/REST APIs: While Pact excels in testing HTTP-based services, its support for other communication protocols (e.g., gRPC, Kafka, or messaging systems) is either limited or requires additional plugins or extensions, making it less flexible for certain use cases.
• Learning Curve: Pact introduces concepts like consumer-driven contracts, contract verification, and the Pact Broker, which may require developers to spend time learning and configuring the system correctly, especially if they are unfamiliar with contract testing.
• Maintaining Contracts: Contracts can become outdated or too rigid over time. If teams do not carefully manage and update contracts as services evolve, it can lead to unnecessary test failures or increased maintenance overhead.
• Tight Coupling Between Consumer and Provider: While Pact provides clear contracts, it can also lead to a form of coupling between consumer and provider teams. Changes to the provider may require changes to the consumer contracts, which could slow down development if not managed carefully.
• Overhead in Managing the Pact Broker: Although the Pact Broker is useful for sharing and managing contracts, it introduces additional infrastructure and setup. Organizations need to maintain and administer the Pact Broker, which adds operational complexity.
• Complexity in Large Systems: In very large, highly interconnected systems with multiple services, managing all the contracts can become complex and challenging, particularly when services are interconnected in a web of dependencies. This complexity might require additional tooling to scale properly.
• Requires Both Consumer and Provider Buy-in: For Pact to work effectively, both the consumer and provider teams need to agree on adopting the contract testing approach. If one side doesn’t buy into the process, the testing framework can lose its value.

Summary

Contract testing plays an important role in allowing you to have confidence in your microservices testing and ensuring that the dependencies still meet the expectations of a service. Utilizing a tool like Pact to help with this makes it easy to write your contract tests and improves your ability to deploy independently across your different services and APIs.