From pans to python: A culinary introduction to multi-threading and multi-processing

Abstract

I recently had to explain to a non-technical friend the concept of multi-threading (concurrent execution) and multi-processing (parallel execution). That's when I came up with a cooking analogy, which I think might be worth sharing.

Although the terms thread and process are often used interchangeably, it's not the same, and it's useful to know how they differ, and when to use which.

I will provide examples illustrating the concepts, and help you get started if you want to write your own multi-threading or multi-processing code in Python.

A cooking analogy

Imagine a CPU core to be a frying pan that at all times only fits a single ingredient. You can use the pan to fry an egg OR you can use the pan to fry bacon (an ingredient is equivalent to a blocking function).

Cooking the egg takes a minute, and frying the bacon takes a minute. Hence, preparing your meal (aka. finishing the execution of your code) takes 2 minutes. This is considered sequential cooking (code) execution.

?Multi-threading makes use of only a single pan for multiple ingredients.

Imagine you swap the bacon and egg quickly in and out of the pan. To bystanders (the user of an application) this looks as if you'd be cooking both ingredients in parallel. This essentially is multi-threading (concurrent execution of your code). As you can tell, it's not the most efficient use of the pan, but if the swap is performed fast enough, you achieve something that looks and feels like parallelism.

Using a single pan is simply less work. Grabbing and then cleaning two, or three, or four pans is more work. You get the point. Therefore, in technical terms, threads are considered more "lightweight" than processes. However, keeping track of the different ingredients (memory-space) and their status might require extra mechanisms to make sure that you exactly know where and in which state every ingredient is (need for synchronization mechanisms).

Multi-processing makes use of an individual pan for every ingredient.

Every ingredient is prepared in its own pan. The benefit is that each pan's task is clear (straightforward to write the code) and that the pan can focus its full energy (computational power) only on frying a single ingredient. This is considered parallel execution.

Obviously cleaning up two pans is more work than one. Additionally, because every pan is fully independent of each other, letting the "egg-pan" know that the "bacon-pan" is done with its task, requires an inter-pan communication mechanism. With two pans this is quite simple, but with 10 pans, the complexity of inter-pan (inter-process) communication becomes less trivial.

Imagine you are having a third pan that fries onion. In order to finish its task, the "egg-pan" needs to add the onions to the egg (move data or state from one process to another). Just a second later the "bacon-pan" requests the onions too, but at that point, the onion pan is already empty. Hence, the "bacon-pan" would never be able to finish its task because it lacks an ingredient (aka. the process is starving; and your guest).

Ensuring that the inter-pan communication and sharing of ingredients is performed correctly calls for dedicated pan (process) synchronization techniques. This is also relevant for threads. Threads mainly use it to prevent race conditions, where as processes primarily use synchronization primitives for sharing state or data.

From pans to python

Sequential execution

To make the above less abstract, let's consider the below code. Preparing the egg and bacon each takes 60 seconds. Hence, two subsequent calls of the function would take 2 minutes to finish. This is an example of sequential code execution.

def pan(ingredient, prep_time=60):
    print(f"Start cooking: {ingredient}")
    time.sleep(prep_time)
    print(f"Finished cooking: {ingredient}")


def main():
    pan("egg")
    pan("bacon")
    

if __name__ == "__main__":
    main()

By using threads and processes, you can run the functions in parallel

Using threads or processes you can theoretically cut down the execution time to 1 minute.

Concurrent execution with threads

This is a simple example using threads which should be sufficient to get you started using threads in Python. You can copy this snippet as is and save it in a file. For example, call the file "my_threads.py". Then execute the file running "python my_threads.py" from your command line.

from multiprocessing import Process
from time import sleep
import threading



def pan(ingredient: str, prep_time: int = 3):
    print(f"Start cooking {ingredient}")
    sleep(prep_time)
    print(f"Finished cooking {ingredient} {prep_time}s later")



def start_threads():
    thread1 = threading.Thread(target=pan, args=("egg", 4))
    thread2 = threading.Thread(target=pan, args=("bacon", 3))


    thread1.start()
    thread2.start()


    print("Starting pan() thread")


    while thread1.is_alive() or thread2.is_alive():
        print("Pan thread running ..")
        sleep(1)


    print("All pan() thread completed")

def main():
    start_threads()



if __name__ == '__main__':
    main()

This is the output

Start cooking egg
Start cooking bacon
Starting pan() thread
Pan thread running ..
Pan thread running ..
Pan thread running ..
Finished cooking bacon 3s later
Pan thread running ..
Finished cooking egg 4s later
All pan() thread completed

Parallel execution with processes

The snippet to use processes rather than threads looks very similar. The only difference is that you instantiate processes instead of thread objects.

from multiprocessing import Process
from time import sleep
import threading



def pan(ingredient: str, prep_time: int = 3):
    print(f"Start cooking {ingredient}")
    sleep(prep_time)
    print(f"Finished cooking {ingredient} {prep_time}s later")



def start_processes():
    process1 = Process(target=pan, args=("egg", 4))
    process2 = Process(target=pan, args=("bacon", 3))


    process1.start()
    process2.start()


    print("Starting pan() processes")


    while process1.is_alive() or process2.is_alive():
        print("Pan processes running ..")
        sleep(1)


    print("All pan() processes completed")



def main():
    start_processes()



if __name__ == '__main__':
    main()

This is the output

Pan processes running ..
Start cooking egg
Start cooking bacon
Pan processes running ..
Pan processes running ..
Pan processes running ..
Finished cooking bacon 3s later
Pan processes running ..
Finished cooking egg 4s later

Conceptually, the output and the code for threads and processes look almost identical. However, execution under the hood differs quite a bit, and it's important to chose the right one for your use case.

The following section helps you decide which of the two you might to want to go with, based on your use case and requirements.

When to use Threads vs. Processes

Threads

Threads are used when a single process needs to appear to be responsive while performing other sporadic tasks concurrently. These other tasks are usually not expected to run for the entirety of the application. Meaning, the logic that the thread executes takes only a few seconds or minutes, whereas your actual application runs for hours or days.

Example Your application is having a graphical user interface (GUI) and displays stock prices. Every minute the prices update. The API call is short-lasting, effectively an I/O action, and not computationally complex. A great candidate for threads.

• Shared memory-space Threads use the same, hence a shared, data and state space. In technical terms: they share a memory space. No dedicated interprocess communication (IPC) is required.
• Synchronization primitives The use of synchronization primitives is most likely required to provide race conditions. Due to the shared memory-space, its easier to implement concurrency based bugs.
• Lightweight Threads have their own stack, but a shared memory space. Hence they are considered "lightweight", compared to processes.
• Single-core Threads run only on a single core at a time. The OS decides on which core to place (schedule) the thread. Multiple threads may, but do not have to run on the same core.
• Intel Hyperthreading / AMD Symmetric Multithreading (SMT) Proprietary technologies aiming to leverage available cores in the CPU. It let threads behave like processes without any additional work by developers. It essentially tries to move threads onto other available cores.
• I/O Operations Well suited for operations that are I/O bound, for example reading files, web requests, etc.

Global Interpreter Lock (GIL) Responsible in Python for giving multiple threads CPU-time and handing associated locks (mutex) and thread states.

Processes

Processes are usually chosen when the logic is expected to run for the entirety of the application. Further, you would choose processes over threads when you need more raw compute power and more granular control over certain aspects of the parallelism.

Example Your application is listening on a specific Socket. This type of operation is usually a blocking and permanent task. Every request that you read from the socket might need it's own processing. This processing of the request itself on the other hand could be handled in a thread.

• Multiple CPU cores Leverage multiple cores in parallel. Do not only switch between threads very quickly, but perform real parallel execution.
• Dedicated memory-space Processes do not share a memory space. Hence, if you want to share data among processes you need to make use of synchronized data structures such as Queues.
• Inter Process Communication (IPC) IPC becomes a necessity as every process has its own dedicated memory space. Threads primarily need to used synchronization primitives to prevent race conditions, whereas for processes it's more of a way to perfor inter-process communication.
• Larger memory footprint Processes have their own memory space and stack, hence are not as lightweight as threads.
• Kill child process Processes are designed to be killable, whereas threads are not intended to be terminated.

Caveat

Specific details of processes and threads might be slightly different in your language of choice.

Fin

Congratulations, you now know the difference between multi-threading (concurrent execution) and multi-processing (parallel execution)!