Spring Boot: How Many Requests Can Spring Boot Handle Simultaneously?

Spring Boot is an essential framework in Java development, providing developers with efficient and user-friendly tools. Consequently, related interview questions are also significant.

Today, let’s examine a classic interview question: How many requests can Spring Boot handle simultaneously?

To be precise, the number of requests Spring Boot can handle simultaneously does not depend on the Spring Boot framework itself but on its embedded web container (since the web container’s behavior determines Spring Boot’s behavior, we can consider the answers to these two questions to be the same).

1. Three Main Web Containers

Currently, there are three main web containers in the market: Tomcat, Undertow, and Jetty.

Among these, Tomcat is the default web container for the Spring Boot framework.

Their differences are as follows:

1.1 Tomcat

Tomcat is an open-source project under the Apache Software Foundation and is one of the most widely used Servlet containers. It fully implements the Java Servlet and JavaServer Pages (JSP) specifications.

Tomcat is not only a Servlet container but also a lightweight application server, although it is considered slightly heavier compared to other lightweight servers.

Tomcat supports numerous enterprise-level features such as SSL, connection pooling, etc., making it suitable for running large, complex enterprise applications.

Its stability and maturity have been proven through years of enterprise-level applications, making it the preferred web container for many enterprises.

1.2 Undertow

Undertow is a flexible, high-performance web server and reverse proxy server developed by Red Hat.

It is the default web container for the WildFly application server. Undertow is designed for low memory usage and high concurrency, excelling at handling large numbers of short connections, such as RESTful API services.

Undertow supports Servlet 3.1, WebSocket, and non-blocking IO (NIO) and is one of the modern servers supporting the HTTP/2 protocol.

Its design philosophy is to provide a modular, embeddable solution that is easy to integrate into existing systems and suitable for microservices architectures.

1.3 Jetty

Jetty is an open-source, lightweight web server and Servlet container maintained by the Eclipse Foundation.

It is known for its embeddability and high configurability, often used in scenarios requiring quick startup and lightweight deployment, such as development phases, test environments, or lightweight applications.

Jetty also supports the Servlet specification and WebSocket and is based on NIO, making it perform well in handling large concurrent connections.

Jetty emphasizes flexibility and extensibility in its design, making it easy to customize through APIs to meet specific needs, thus being popular in cloud environments, continuous integration, and DevOps.

In summary, Tomcat is suitable for large applications due to its maturity and enterprise-level features;

Undertow excels in high-performance and low-memory usage, especially for high-concurrency short connection scenarios;

and Jetty is characterized by its lightweight, flexibility, and ease of embedding, suitable for rapid development and lightweight deployment.

2. Maximum Number of Connections and Maximum Waiting Number

Taking Tomcat, the default web container for the Spring Boot framework, as an example, the number of requests it can handle simultaneously is configured in the spring-configuration-metadata.json file in the Spring Boot framework, as shown below:

Open this file and search for ‘server.tomcat.max-connections’ (Tomcat’s maximum connections) to find the following result:

This means that by default, Tomcat allows a maximum of 8192 connections (8192 = 8 * 1024).

At this point, one might think, ‘By default, Spring Boot can handle 8192 requests simultaneously.’ If you think so, you are mistaken. Why?

Because, although Tomcat can allow a maximum of 8192 connections, Tomcat also has a maximum waiting number, which means that if it reaches 8192, there is a waiting queue that can store the requests’ connections.

So, the number of connections Spring Boot can handle simultaneously is equal to Tomcat’s maximum connections plus Tomcat’s maximum waiting number.

So, what is the maximum waiting number?

We continue in the spring-configuration-metadata.json file and search for ‘server.tomcat.accept-count’ (Tomcat’s maximum waiting number). The search results are shown below:

3. Concurrent Request Handling

Therefore, we conclude: By default, the number of requests that Spring Boot can handle simultaneously = maximum connections (8192) + maximum waiting number (100), resulting in 8292.

Of course, these two values can be modified in the Spring Boot configuration file, as shown in the configuration below:

server:
  tomcat:
    max-connections: 2000 # Maximum connections
    accept-count: 200 # Maximum waiting number        

4. Additional Knowledge: Setting Up the Web Container

How do you set the web container to Jetty or Undertow in the Spring Boot framework? Let's take a look.

4.1 Setting the Container to Jetty

To set the Spring Boot framework's web container to Jetty, simply modify the pom.xml file as shown below:

<dependencies>
    <!-- Spring Boot Starter Web but exclude Tomcat -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
        <!-- Exclude Tomcat -->
        <exclusions>
            <exclusion>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-tomcat</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <!-- Add Jetty starter dependency -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-jetty</artifactId>
    </dependency>
</dependencies>        

In other words, simply exclude the default Tomcat and add the Jetty dependency.

4.2 Setting the Container to Undertow

To set the Spring Boot framework's web container to Undertow, the approach is similar to the Jetty implementation above. Just modify the pom.xml file as shown below:

<dependencies>
    <!-- Spring Boot Starter Web but exclude Tomcat -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
        <exclusions>
            <exclusion>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-tomcat</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <!-- Add Undertow starter dependency -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-undertow</artifactId>
    </dependency>
</dependencies>