TestContainers and Spring Boot

Introduction

Almost every enterprise application that we build is communicating with external service. Some of the common integration's include Message Brokers ( such as Kafka, RabbitMQ etc.. ), Databases and External API’s. The business logic that the service is responsible for is integrated with such services.

Unit/Integration tests are a MUST for shipping a good software. Not only they prevent future bugs but they also ensure that our code can be executed successfully for some predefined cases. As the application becomes more complex writing tests also becomes more complex.

TestContainers is a powerful technology that enables the developers to test the entire flow in their test cases. TestContainers is built on top of Docker and it is a technology that can create Docker Containers when the tests are started and destroy the containers once the tests have finished. For some common images there are already pre-configured test containers. It can run and manage containers from any docker image.

The Problem

Let’s say we need to build an application that will be used for enrolling students in a university.

We already have an application where a student can apply for a university, from that service we will publish a message through Kafka and the new service that needs to be built needs to listen to the Kafka topic and when there is a new message process it. The processing of the message first needs to validate the student’s GPA with an API call to Validator Service. If the student’s GPA is valid he will be saved in the database otherwise he won’t be saved.

The development part seems pretty easy the steps could be defined such as:

• Create a Kafka Listener
• When a Kafka Message is received, call the Validator Service
• Take the response from Validator Service

However once you start thinking about writing Integration tests things get complicated.

• How would I test my Kafka Listener if it is triggered when there is a new message in the Queue?
• How would I call the Validator Service from the tests?
• In which database am I going to store the student?

Available Solutions

In a real world application the integration’s between our application and some external service always grows. In the beginning this application might be interacting with 3 external services ( Kafka,Database and external API) but in the future we might add new integration’s ( e.g KeyCloak for authentication and so on).

We have 2 choices for setting our unit/integration tests. We could either go with mocking the external services (Mockito) and use H2 in-memory database for testing or use TestContainers and not mock anything!

Let’s see how much coverage they provide.

We will use the following colors to identify what is being mocked and what isn’t.

RED→ Mocked

BLUE→ Real Integration

TestContainers

As mentioned above, TestContainers is a powerful technology that will enable us to test our application with real dependencies. We will spin up a Kafka instance, PostgreSQL Database and the external API ( Validator Service ), run all the tests and then shut down everything.

Here we can identify 3 external dependencies for which we will use TestContainers to spin containers.

• Kafka
• Validation Service
• PostgreSQL Databases

Here the whole flow is tested with real dependencies.

Mockito + H2 In-Memory Database

The second approach is to use Mockito and H2 In-Memory Database. This approach has several drawbacks. We cannot test the end-to-end flow so we have to mock the Kafka part, we have to mock the API call to the Validation Service and we need to use a different database ( H2 ) instead of the real one that we will be using on our live environment.

Here we can see the following problems :

• We cannot produce/consume messages
• We cannot call the Validation Service – options are either Mockito or mock HTTP server ( such as WireMock)
• Database will be H2 instead of PostgreSQL ( This is crucial if the application is using native database specific queries )

Key Benefits of Using TestContainers

1. Realistic Testing Environment: TestContainers allows us to test with the same dependencies we use in production, reducing the mismatches between testing and live environments.
2. Comprehensive Coverage: It provides end-to-end testing capabilities, ensuring that all components of the system interact correctly.
3. Ease of Setup: With pre-configured containers for common services, setting up and tearing down environments becomes effortless.

Limitations of Mockito + H2 Approach

1. Limited Integration Testing: Mocking services means we cannot fully test the integration points, potentially leading to missed issues.
2. Different Database Behavior: Using H2 instead of PostgreSQL can lead to differences in query behavior and missed database-specific issues.
3. Maintenance Overhead: Managing mocks and maintaining their accuracy as the application evolves can become a headache.

Integration with Spring Boot

Integrating TestContainers and Mockito in a Spring Boot application is pretty simple. First we need to add the correct dependencies in our pom.xml ( if we are using Maven ) and then start building the tests.

Dependencies for TestContainers

<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-testcontainers</artifactId>
	<scope>test</scope>
</dependency>
<dependency>
	<groupId>org.testcontainers</groupId>
	<artifactId>junit-jupiter</artifactId>
	<scope>test</scope>
</dependency>
<dependency>
	<groupId>org.testcontainers</groupId>
	<artifactId>kafka</artifactId>
	<scope>test</scope>
</dependency>
<dependency>
	<groupId>org.testcontainers</groupId>
	<artifactId>postgresql</artifactId>
	<scope>test</scope>
</dependency>        

Then we start to build the tests! I won’t cover the business logic here ( the real Kafka Listener and the Validation Service, if you want to take a look at that, the code is available on GitHub and repository is shared on the bottom).

Tests

TestContainers

For TestContainers to know which containers we would like to spin during our tests, we need to define a Configuration class.

@TestConfiguration(proxyBeanMethods = false)
public class TestcontainersConfiguration {

	@Container
	private static final GenericContainer<?> gpaValidatorContainer = new GenericContainer<>(DockerImageName.parse("gpavalidator:latest"))
			.withExposedPorts(8081);

	@Container
	private static final KafkaContainer kafkaContainer = new KafkaContainer(DockerImageName.parse("confluentinc/cp-kafka:latest"));

	@Container
	private static final PostgreSQLContainer<?> postgresContainer = new PostgreSQLContainer<>("postgres:latest");

	@Bean
	@ServiceConnection
	public KafkaContainer kafkaContainer() {
		return kafkaContainer;
	}

	@Bean
	@ServiceConnection
	public PostgreSQLContainer<?> postgresContainer() {
		return postgresContainer;
	}

	@PostConstruct
	public void startGpaValidatorContainer() {
		if(!gpaValidatorContainer.isRunning()) {
			gpaValidatorContainer.start();
		}
		String baseUrl = "https://" + gpaValidatorContainer.getHost() + ":" + gpaValidatorContainer.getMappedPort(8081);
		System.setProperty("validator.baseUrl", baseUrl);
	}
}        

Here we are telling TestContainers that we need 3 Containers for this application Kafka, PostgreSQL and GpaValidator.

Spring Boot and TestContainers have auto configuration for Kafka and PostgreSQL, so our application will be connected to these containers automatically without a single line of configuration!

Our custom container ( which is the Validator Service ), we need to set the base url.

Let’s review the test cases.

KafkaConsumerServiceTest

@ContextConfiguration(classes = TestcontainersConfiguration.class)
@SpringBootTest
@Testcontainers
@ActiveProfiles("testcontainers")
public class KafkaConsumerServiceTest {
    @Autowired
    private StudentRepository studentRepository;

    @Autowired
    private KafkaSenderService kafkaSenderService;

    @Test
    public void testKafkaListenerWithLowGpa() throws InterruptedException {
        StudentDTO message = new StudentDTO();
        message.setFirstName("John");
        message.setLastName("Doe");
        message.setEmail("[email protected]");
        message.setGpa(1.5);

        kafkaSenderService.sendStudentDTO(message);

        await().atMost(5, TimeUnit.SECONDS).untilAsserted(() -> {
            assertThat(studentRepository.count()).isEqualTo(0);
            assertThat(studentRepository.findByEmail(message.getEmail())).isEmpty();
        });
    }

    @Test
    public void testKafkaListenerWithHighGpa() throws InterruptedException {
        StudentDTO message = new StudentDTO();
        message.setFirstName("Mark");
        message.setLastName("Doe");
        message.setEmail("[email protected]");
        message.setGpa(3.5);

        kafkaSenderService.sendStudentDTO(message);

        await().atMost(5, TimeUnit.SECONDS).untilAsserted(() -> {
            assertThat(studentRepository.count()).isEqualTo(1);
            assertThat(studentRepository.findByEmail(message.getEmail())).isNotEmpty();
        });
    }

    @AfterEach
    public void tearDown() {
        studentRepository.deleteAll();
    }
}        

Here in this test class there are few important things :

• We use the @ContextConfiguration annotation to pass the TestContainers configuration
• We use the @TestContainers annotation to let TestContainers manage our containers
• Then we have the first and the second test that uses KafkaSenderService to send a message to Kafka. We expect that that message will be received by our Kafka Listener then the Validation Service will be called and based on the GPA it will be saved or not saved in the Database.

StudentRepositroyTest

In this test we will showcase the difference between using PostgreSQL and H2. We will use the RANK() function which behaves differently in PostgreSQL and H2. In PostgreSQL we will get the ranked students ordered while in H2 the ranking will be the same but the order is not the same.

We have the following native query :

@Query(value = "SELECT s.id, s.first_name AS firstName, s.last_name AS lastName, s.email, s.gpa, " +
        "RANK() OVER (ORDER BY s.gpa DESC) AS rank " +
        "FROM students s", nativeQuery = true)
List<StudentWithRankProjection> findStudentsWithRankByGpa();        

Then we have the following test class :

@SpringBootTest
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
@ContextConfiguration(classes = TestcontainersConfiguration.class)
@ActiveProfiles("testcontainers")
//@ActiveProfiles("h2")
class StudentRepositoryTest {

    @Autowired
    private StudentRepository studentRepository;

    @BeforeAll
    void setUp() {
        Student student1 = new Student();
        student1.setFirstName("John");
        student1.setLastName("Doe");
        student1.setEmail("[email protected]");
        student1.setGpa(3.5);
        studentRepository.save(student1);

        Student student2 = new Student();
        student2.setFirstName("Jane");
        student2.setLastName("Doe");
        student2.setEmail("[email protected]");
        student2.setGpa(3.8);
        studentRepository.save(student2);

        Student student3 = new Student();
        student3.setFirstName("Jim");
        student3.setLastName("Beam");
        student3.setEmail("[email protected]");
        student3.setGpa(3.2);
        studentRepository.save(student3);
    }

    /*
        This test is used to demonstrate that the native query from StudentRepository which is using the
        RANK() function is not working the same in PostgreSQL and H2.
        It is correctly ranking the students but the order is different.
     */
    @Test
    public void testFindStudentsWithRankByGpa() {
        List<StudentWithRankProjection> rankedStudents = studentRepository.findStudentsWithRankByGpa();

        assertThat(rankedStudents).isNotNull();
        assertThat(rankedStudents.size()).isEqualTo(3);

        assertThat(rankedStudents.get(0).getRank()).isEqualTo(1);
        assertThat(rankedStudents.get(0).getGpa()).isEqualTo(3.8);

        assertThat(rankedStudents.get(1).getRank()).isEqualTo(2);
        assertThat(rankedStudents.get(1).getGpa()).isEqualTo(3.5);

        assertThat(rankedStudents.get(2).getRank()).isEqualTo(3);
        assertThat(rankedStudents.get(2).getGpa()).isEqualTo(3.2);
    }
}        

We have the following observations here:

• The @BeforeAll method saves 3 students with different GPA in our database.
• The testFindStudentsWithRankByGpa fetches the students using the native query and checks the rank,GPA and the order.
• Using TestContainers this test is successful. However switching to H2 this test fails. ( you can switch to H2 by commenting out the ActiveProfile with TestContainers and the TestContainers configuration and uncomment the H2 profile.

Be careful when using native queries. I personally always prefer using Criteria API/JPQL/HQL or any database agnostic queries but sometimes you must use native queries and that is okay. Just be aware of the differences in your test cases if you are using H2.

Mockito

With Mockito we will aim to cover the same tests and see the differences.

KafkaConsumerServiceTest

@ExtendWith(MockitoExtension.class)
public class KafkaConsumerServiceTest {

    @Mock
    private StudentRepository studentRepository;

    @Mock
    private StudentMapper studentMapper;

    @Mock
    private ValidateGpaHttpService validateGpaHttpService;

    @InjectMocks
    private KafkaConsumerService kafkaConsumerService;

    @Test
    public void testListenWithHighGpa() {
        StudentDTO message = new StudentDTO(1L,"Jane", "Doe", "[email protected]", 3.5);
        StudentValidationInput input = new StudentValidationInput(message.getFirstName(), message.getLastName(), message.getGpa());
        Student student = new Student();

        when(validateGpaHttpService.validateGpa(input)).thenReturn(true);
        when(studentMapper.studentDTOToStudent(message)).thenReturn(student);

        kafkaConsumerService.listen(message);
        verify(studentRepository, times(1)).save(student);
    }

    @Test
    public void testListenWithLowGpa() {
        StudentDTO message = new StudentDTO(1L,"Jane", "Doe", "[email protected]", 1.5);
        StudentValidationInput input = new StudentValidationInput(message.getFirstName(), message.getLastName(), message.getGpa());

        when(validateGpaHttpService.validateGpa(input)).thenReturn(false);
        kafkaConsumerService.listen(message);
        verify(studentRepository, never()).save(any(Student.class));
    }
}        

In this test class we can notice the following things:

• We use @Mock and @InjectMocks annotation’s to mock our components.
• In our test cases we use when() method to mock certain behavior such as
• We are not testing the Kafka Integration, so we are calling this as a standard java method ( kafkaConsumerService.listen ) so we cannot be sure if our Kafka Listener actually listens to messages from Kafka.
• Even changing the GPA to any value, this tests will pass since Validator Service is also mocked

StudentRepositoryMockitoTest

In this class we will review the same native query we used for PostgreSQL and H2 but this time built with Mockito mocking the query call to the database.

@ExtendWith(MockitoExtension.class)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
public class StudentRepositoryMockitoTest {
    @Mock
    private StudentRepository studentRepository;

    private List<StudentWithRankProjection> mockRankedStudents;

    @BeforeAll
    void setUp() {
        StudentWithRankProjection student1 = mock(StudentWithRankProjection.class);
        when(student1.getRank()).thenReturn(1);
        when(student1.getGpa()).thenReturn(3.8);

        StudentWithRankProjection student2 = mock(StudentWithRankProjection.class);
        when(student2.getRank()).thenReturn(2);
        when(student2.getGpa()).thenReturn(3.5);

        StudentWithRankProjection student3 = mock(StudentWithRankProjection.class);
        when(student3.getRank()).thenReturn(3);
        when(student3.getGpa()).thenReturn(3.2);

        mockRankedStudents = Arrays.asList(student1, student2, student3);
    }

    @Test
    public void testFindStudentsWithRankByGpa() {
        when(studentRepository.findStudentsWithRankByGpa()).thenReturn(mockRankedStudents);

        List<StudentWithRankProjection> rankedStudents = studentRepository.findStudentsWithRankByGpa();

        assertThat(rankedStudents).isNotNull();
        assertThat(rankedStudents.size()).isEqualTo(3);

        assertThat(rankedStudents.get(0).getRank()).isEqualTo(1);
        assertThat(rankedStudents.get(0).getGpa()).isEqualTo(3.8);

        assertThat(rankedStudents.get(1).getRank()).isEqualTo(2);
        assertThat(rankedStudents.get(1).getGpa()).isEqualTo(3.5);

        assertThat(rankedStudents.get(2).getRank()).isEqualTo(3);
        assertThat(rankedStudents.get(2).getGpa()).isEqualTo(3.2);
    }
}        

Here we can see the following key parts:

• In the setUp method we are mocking the students
• In the test case we are mocking the native query using when() method to return our mocked students any time that query is called.
• Then we fetch the mocked students and validate the results.

While this test passes successfully and looks like a valid test case it actually doesn’t test anything. Even if we have invalid SQL in the native query this will pass since that query will never be executed and the mocked students will be returned whenever we call it.

Be careful what you are mocking as it can lead to issues/bugs in live environments that could have been observed in the tests if they were written properly.

Conclusion

In conclusion, while both approaches - using TestContainers and the combination of Mockito with an H2 in-memory database – are possible, TestContainers provides a significant advantage for complex integration testing. By leveraging real instances of the external dependencies ( in this case Kafka, PostgreSQL, and the Validator Service), TestContainers ensures that our tests are more realistic and closer to the production environment. This approach minimizes the risk of encountering integration issues in production that were not caught during testing.While Mockito and H2 are useful for simpler scenarios and quicker tests, TestContainers is the preferred approach for robust, enterprise-level applications requiring thorough testing of all integration points.

TestContainers not only enhances our testing strategy but also contributes to higher quality and reliability of the software we deliver, leading to better user satisfaction and reduced maintenance costs in the long run.

The code can be found at the following GitHub repository : https://github.com/petrovskimario/Spring-Boot-with-TestContainers

Happy Coding!