Damn, Threads in web server too !!

?? Chronological learning outcomes from this article. Important low level and high level details about the functioning of:

?? Single threaded web servers

?? Multi threaded web servers

?? Multi Processor web servers

[?] Scenario:

?? When you sit on www.youtube.com and try to checkout the interview experience of 1.2 & 1.5 Cr packages for freshers on different ed-tech channels. You essentially request a web server for a response (HTML page etc.) for the website (YouTube in this case) itself.

Let's see how the web server serves the response in different models of the web server based on HOW the request is processed.

[?] Engineering how the single threaded web server works:

Note: I've omitted the load-balancer layer from this diagram knowingly

??Background:

There is an Apache HTTP Server process running on port 80 in your web server (web server just means a dabba or a computer). This process keeps listening to incoming web requests on the port 80. Port 80 is the default port for TCP connections over HTTP.

??Chronological Algorithm:

1. Client makes a request on the port 80.
2. A single thread (from the Apache HTTP Server) keeps listening to this port (unless explicitly stopped) and grabs the TCP socket connection here.
3. Now due to the web server being single threaded in nature, it processes the request in the socket connection via the SAME thread which ACCEPTED the connection from the client in the first place.
4. At this stage, if Client-2 or Client-3 make a request to the SAME web server on the port 80, then they would be BLOCKED, because the ONLY thread which was there to accept the socket connection, is BUSY processing the previous client requests.
5. When the client request processing is finished, then this thread resumes listening to the same port and client-2/3 are served subsequently.

?? Unsolicited commentary:

?? This model seems to have a major con of being BLOCKING in nature. Successive client requests would get blocked due to a single client request (possibly taking indefinite amount of time to get processed) rendering your system as a Non Highly Available System.

?? If you've understood this drawback, then it won't be hard for you to wrap your head around the simple solution to this.

TL;DR - Scroll Down!

[?] Engineering how the Multi Threaded Web Server works:

?? As opposed to previous design, the multi threaded web server enforces that, the thread which accepts the connection from the client via port 80, essentially dispatches / delegates the processing of the request to one of the worker threads.

?? This essentially FREEs up the connection-accepting-thread to concurrently serve clients in parallel. Hence reducing the probability, that the web server becomes unavailable due to the thread busyness.

?? Note: There can be different ways of managing the worker threads using Thread Pools, Executor Service, Custom Thread Pool etc.

???? All the threads that get spawned from the process share the same virtual memory address space, except the individual thread stack that's scoped at the thread level.

???? Hence the parent-process CAN share some crucial metadata with all individual threads via the heap, global memory, cache etc. Refer my previous article if this is not clear !!

[?] Engineering how the Multi Processor Web Server works:

?? Please note that in the previous solution, there are multiple threads being spawned from just 1 process which is listening to the port 80.

Q: In this approach, could we have multiple processes listening to the same TCP port 80 instead of spawning multiple threads from within a process?

?? For a TCP connection,?you can ONLY have one application / process listening on the same port at one time. Unless you have 2 network cards, and the first process listens on the first IP and second process on the second IP using the same port number.

?? For UDP connection however, multiple applications / processes can subscribe to the same port.

??Functioning of multi processor web server:

?? That being said, for spinning up a web server that uses multiple processes to serve the clients. You could technically spawn multiple child processes from the parent process.

?? And then you can have the same model of multiplexing the client requests to these child processes for processing.

??3 disadvantages of having a multi processor web server as opposed to multi threaded web server

?? Threads are much less expensive than spawning processes. Hence the number of parallel clients that your web server can handle would get directly affected with this thought.

?? The port 80, may end up having contention due to multiple processes trying to listen there.

?? We would need IPC (Inter Process Communication) to share some metadata with all child-processes for serving the clients. This can easily be done for the multi threaded web server, via the shared heap - since that's common to all threads spawned from within a process.

?? Hence for these reasons, a multi threaded web server can PERHAPS be a better bet for higher scalability and performance as opposed to a multi processor web server.

?? ???Please comment if you have any questions or found something incorrect !!

?? Godspeed!