Some reasoning over using threads based web servers
Well Known that widely-used programming languages supports threading model (multi-threading), such as Java, C++, Python, C#, etc.
The threading model or so named "thread-per-request" model is a traditional approach to handling web requests in synchronous programming models where each incoming request to the WEB server is handled by a dedicated thread. Just this thread is responsible for complete processing the request and composing response. Two main threading models are most famous currently: Platform Threading Model and Virtual Threading Model.
This article has a goal to show out how using threading models in Web servers and evaluating them functional possibilities. Here the Web servers on Java platform will be evaluating only by using Ubuntu 20.04 system.
Platform Threading Model
On a thread based system the web server creates a new thread as soon as a request receiving. So all the work for this request is handled by that thread. Such thread is named often as platform or real thread due to its implemented as a thin wrapper around an operating system (OS) thread. Every thread exists until his work will not be finished, independently it's in waiting state or uses processor resource.
This means, that the maximum of available threads on a computer determines how many concurrent users can send request to the web server simultaneously.
Let say, Tavr – average processing time of request and RQS – requests per second.
Therefore RSS = 1/ Tavr is average responses per second.
If RQS < RSS then all is OK and server works quite calmly, the number of threads is not growing. The minimum count of threads are enough to be processing every request.
But a dangerous situation could occur in case of RQS > RSS. In this case the number of threads will growing and can reaching critical state when machine resources are overhead.
The number of threads during some time can be evaluated by using simple formula:
Nt = No + (RQS - RSS)*t
where Nt – current number of threads
No – initial number of threads
t - observation time
Thus, for instance, if No=10 (initial threads pool) and (RQS – RSS) = 10 Req/Sec then in 10 min (600 sec) the count of requests processing threads will be already above 6000!
Sure, above mentioned formula is abstract and idealized representation of real situation but anyway it gives some insights about existing problem concerning usage of threading model.
Take into account also that every thread is in fact a lightweight process and naturally as many processes are working in one machine then more slowly they working because of competition for machine resources.
Note also than the maximum count of threads in any machine is limited. In practical terms, this limit is influenced by the operating system and the available system resources, such as memory and CPU cores. So, the only way to rectify such situation is scaling (horizontal or vertical).
Let’s to do some experiment and use for instance a simple HTTPS server build on Java platform.
The certificate for HTTPS server can be generated like described in article. Sure, of course. we can use JKS certificate because many Java developers are still used it. But notice that PFX format certificate (PKCS12 KeyStore) is recommended to use now in Java (since JDK 17.x) instead of deprecated JKS KeyStore. Therefore we will use the PFX format, given that in the current test we are using JDK 19.x.
Start the server and evaluate the benchmark.
To evaluate the performance, we will be used autocannon tools to benchmark test our HTTPS server (results of test shown below):
HTTPS Server receives quite big of client requests during 10 sec. Active threads count evaluates every 1 sec.
Above depicted test results shows that during 10 sec HTTPS server was attacked with speed 2076.9 Req/Sec in average by using 500 connections (users). The average respond latency was 240 ms. Note that about 60 requests was finished with errors (not correctly answered). The requests service threads count was growing quite speedy and reach up to 2252 threads in 10 sec.
Anyway, let’s try to highlight some widespread advantages and disadvantages the threading model.
Advantages:
领英推荐
Disadvantages:
Virtual Threading Model
Oracle company introduced recently so named virtual threads in Java (preview feature since 19.x). This functionality eliminates some of the disadvantages associated with traditional threads based servers. Virtual threads are an abstraction layer between operating-system processes and application-level concurrency which is ideal for high-concurrency scenarios without significant resource overhead. They can lead to practically unbounded resources usage in highly concurrent web servers where thread creation is a bottleneck.
Above mentioned resource contains also the WEB server script (ServerV.java) that used virtual threading model.
So we can start similarly the server by using the following command:
Note that java 19.x virtual threads is in preview state for now. Thus we should use –enable-preview parameter in start command to be used this functionality.
To see the performance, we will be used again autocannon tools to benchmark test our HTTPS server (results of test shown below):
During test no any real thread (OS threads) is used for processing requests and preparing responses.
Above depicted test results shows that during 10 sec HTTPS server has been servicing 500 connections (users) and receiving about 560.9 Req/Sec in average. Throughput was reaching to 636 KB/sec in average. The average respond latency was 1334.5 ms. Note also that no any requests was finished with errors.
Let’s try to highlight some advantages and disadvantages of virtual threading model in java.
Advantages:
Disadvantages:
Notice that although Java Virtual Threads offer significant advantages in terms of scalability, memory efficiency, and simplification of concurrent programming models, however, they also introduce challenges, particularly in terms of performance for CPU-bound tasks, the impact on garbage collection, and the need for updated debugging support. Developers should carefully consider these factors when deciding to adopt virtual threads in their applications.
Short comparison
Both threading models tests averaged results are depicted in table below.
According tests the performance of Virtual Threading Model is unfortunately weak enough but anyway this model has one big benefit - it gives unbounded count of threads and therefore doesn't lead to critical resources consumption and system instability.
Summary
In conclusion, can be say the above mentioned doesn't mean Platform Threading Model shouldn't be used at all. We tried to mark only some problematic places while using such model. Anyway note that threading model in WEB servers development offer significant advantages in terms of performance, responsiveness, and resource utilization. However, it also introduces complexity and potential issues related to synchronization, concurrency and resources usage.
Take into account also that Virtual Threading Model are a big change under the hood, and they are intentionally easy to apply to an existing codebase. Popular WEB Servers like Tomcat already allow for virtual threads (since 10.1.x). GlassFish server (since 7.0.13) is also ready to be supporting virtual threads. Jetty Web Server is also supporting Virtual Threading model since 10.0.x. Applications running on these servers will get scalability gains without any changes to the code, which could have enormous implications for large-scale applications.
So, developers need to carefully consider the trade-offs and use both threading model judiciously to achieve the desired results.