Effortless .NET Integration Testing - Introducing TestContainers

Writing integration tests doesn't have to be hard. Learn how the .NET library can help you use Docker to power your tests.

Introduction

During my years in development, writing integration tests are usually cumbersome and hard.

Just to mention the pain points of traditional integration testing:

• Environment Inconsistencies: The classic “works on my machine” issue, where tests pass locally but fail in other environments.
• Setup and Tear Down Complexity: Creating and maintaining test environments that mimic production can be cumbersome.
• Test Data Management: Ensuring consistent and relevant test data across multiple tests can be tricky.
• Slow Test Execution: Integration tests often run slower than unit tests, leading to longer feedback loops.
• Interdependent Tests: Tests that depend on external services or databases can cause flakiness and hard-to-reproduce failures.
• Resource Intensive: Running multiple integration tests can consume significant resources, impacting other builds or services.

Now, picture a world where your integration tests run seamlessly, thanks to the synergy of .NET and Docker. This is what this article is all about!

Tests that run on containers

This is where Docker steps in. By containerizing your testing environment, you achieve consistent, isolated setups for every test run. Running containers is flexible and good way to achieve, otherwise too difficult, isolation and repeatability of the tests.

Of course, as with everything, setting up Docker can be cumbersome:

• Initial Learning Curve: Understanding Docker concepts like images, containers, volumes, and networks can take some time, especially if you’re new to containerization.
• Environment Configuration: Setting up Docker on different operating systems can lead to compatibility issues. This includes configuring Docker Desktop on Windows, macOS, and Linux.
• Complex Networking: Networking between containers and the host system can be tricky. Ensuring that the right ports are exposed and that containers can communicate with each other securely and effectively can be a challenge. -> It is not once that I spent hours figuring out which port to configure and why host machine is actively refusing it!
• Managing Persistent Data: Ensuring that data persists across container restarts, or properly setting up volumes, requires additional configuration. And this particularly relates to the .NET test libraries with fixtures and collections.
• Orchestration Tools: While Docker Compose simplifies running multi-container applications, learning it and tools like Kubernetes for more complex setups introduces more complexity.
• Debugging: Diagnosing issues inside a container can be more complicated compared to a local development environment. This includes accessing logs and understanding container states.

All this complicate things and discourages an ordinary developer to write tests - even when Docker is around.

TestContainers library

TestContainers.NET is a handy tool for integration testing in .NET. It allows you to start and stop Docker containers directly from your test code, simplifying the setup of dependencies like databases and other services. By running your tests in isolated containers, you avoid conflicts and ensure consistency across different environments. This setup helps you avoid the "works on my machine" problem by providing a reproducible and consistent environment for each test run. With containers, your tests run faster and more efficiently, allowing for parallel execution without any worries about interfering states.

Here’s a quick example of how to use TestContainers in your tests:

using DotNet.Testcontainers.Builders;
using DotNet.Testcontainers.Containers;
using Xunit;

public class MyIntegrationTests
{
    [Fact]
    public async Task TestDatabaseConnection()
    {
        var dbContainer = new ContainerBuilder()
            .WithImage("mcr.microsoft.com/mssql/server")
            .WithPortBinding(1433)
            .Build();

        await dbContainer.StartAsync();

        // Your test logic here
        
        await dbContainer.StopAsync();
    }
}
        

Here is some docus regarding this library:

• Testcontainers
• testcontainers-dotnet/examples/Flyway/tests/Flyway.Tests/DbFixture.cs at develop · testcontainers/testcontainers-dotnet · GitHub
• Testing Using Testcontainers for .NET and Docker - Code Maze (code-maze.com)

If you're using xUnit test framework, you can use IClassFixture and ICollectionFixture helper interfaces to further optimize your code. This is how my initial integration test project structure, based on TestContainers and fixtures, looks like:

For instance, I'm using a Fixture-based class that implements IAsyncLifetime interface to initialize itself asynchronously. This class initialize a container running MS SQL Server instance, configures database, and prepares database context class by using EF Core:

using Microsoft.Data.SqlClient;
using Testcontainers.MsSql;

namespace MyProject.IntegrationTests.Configuration;

public sealed class TestMsSqlContainerFixture : IAsyncLifetime
{
    private readonly MsSqlContainer _container;
    private const string DatabaseName = "MyProjectDB";

    public string ConnectionString { get; init; }

    public TestMsSqlContainerFixture()
    {
        _container = new MsSqlBuilder()
               .WithImage("mcr.microsoft.com/mssql/server:2022-latest")
               .WithCleanUp(true)
               .Build();

        _container.StartAsync().Wait();

        using (var connection = new SqlConnection(_container.GetConnectionString()))
        {
            connection.Open();
            using var command = new SqlCommand($"CREATE DATABASE {DatabaseName};", connection);
            command.ExecuteNonQuery();
        }

        ConnectionString = new SqlConnectionStringBuilder(_container.GetConnectionString())
        {  InitialCatalog = DatabaseName   }.ToString();
    }

    public async Task InitializeAsync()
    {
        await Task.Run(() =>
        {
            var configuration = TestHelper.GetConfiguration();
            using var context = TestHelper.GetDbContext(ConnectionString);
            DbInitializer.CreateSchemaAndSeedData(context, configuration);
        });
    }

    public async Task DisposeAsync()
    {
        await _container.StopAsync();
        await _container.DisposeAsync();
    }
}        

I'm also using Collection and Trait test attributes to further make tests configurable - thus making them more isolated and filtered (see DevOps):

[Collection("Persistence Tests"), Trait("Environment", "Docker"), Trait("Container", "MsSql")]
public class MsSqlContainersTests(TestMsSqlContainerFixture fixture) : IClassFixture<TestMsSqlContainerFixture>
{
    [Fact]
    public async Task TestService_CreateAsync_OperationValid()
    {
        // Arrange 
        using var context = TestHelper.GetDbContext(fixture.ConnectionString);
        ...
        Assert.True(context.ChangeTracker.HasChanges());
        context.SaveChanges();
        Assert.False(context.ChangeTracker.HasChanges());
       ...

        // Act
        var result = await service.SaveAsync(model);

        // Assert
        ...        

Test execution is in minutes, but it works - the only thing you should keep an eye is memory and CPU consumption! The rest (starting and tearing down containers) is done for you! :)

DevOps integration

The great part of this story is that these tests, although written as integration tests, may be part of the build process - which is the thing specific to the unit tests - as they don't have any dependencies to the physical resources. In other words: as everything is containerized - there are no need for preparing and reconfiguring real resources!

TestContainers easily integrate with Azure Pipelines, GitHub Actions, or GitLab CI/CD cycle:

Continuous Integration - Testcontainers for .NET

The tests are being integrated with Azure DevOps process, and I included it into an existing build pipeline with the standard .NET Core pipeline task:

One to notice though is that I separated the execution of integration tests by Trait attribute. As forementioned, this is a good principle for filtering tests, especially if memory and CPU consumption is critical. For instance, Azurite containers use up to 100 GB of memory, which is quite problematic, so I run this tests sequential and separate of other types of tests.

Summary

Using TestContainers library for integration tests in .NET offers several advantages over relying solely on Docker or local resources.

Firstly, TestContainers simplifies the setup and teardown of Docker containers directly within your test code. This eliminates the need for separate scripts or manual configuration, making it easier and faster to define and manage test environments. Each test runs in a clean, isolated container, ensuring consistent and reproducible results, which reduces the "works on my machine" problem.

Secondly, TestContainers automates the lifecycle of Docker containers. It starts containers before tests run and ensures they are properly cleaned up afterward. This means you don’t have to worry about lingering states or resources, leading to more reliable tests and fewer surprises.

Furthermore, by using TestContainers, you can easily integrate multiple services (like databases, message brokers, etc.) into your tests without complex orchestration. It’s ideal for testing interactions between various components of your system.

In short, TestContainers makes integration testing more straightforward, reliable, and scalable, leveraging the benefits of Docker while abstracting away much of its complexity.

This allows you to focus on writing meaningful tests rather than managing infrastructure. So, the developers (and/or QA personnel) are more productive and they keep their focus.

I hope this gave you a brief but useful overview about using TestContainers library in .NET for writing integration tests. Kind regards!