What is parallelism and how does it work

Previous posts covered asynchronous execution where we pass tasks to external resources and use the waiting time to do other useful work.

Now, this series will focus on parallelism. Parallelism is effective for CPU-bound operations, where the workload can be divided and run across multiple cores. We’ll dive deeper into how it works, where it can improve performance, and how to apply it in .NET. This approach used when there are no external resources involved. Instead, we maximize the utilization of the current CPU through parallel execution, ensuring that our system runs at full efficiency.

Async execution vs parallelism

Async execution - it's about waiting for external resources (like a database or a file system) without blocking the main thread. While waiting, the thread is free to do other tasks. Async is ideal for I/O-bound tasks where the CPU doesn't have much work but is waiting for a response.

Meanwhile, in parallelism, there are no external resources involved. Instead, we use CPU cores by dividing work into smaller tasks and running them simultaneously across different threads or processors. This is ideal for CPU-bound tasks, where the CPU is actively doing heavy calculations. Parallelism helps fully utilize CPU power by performing multiple tasks at the same time.

How does parallelism work

In .NET, threads are the building blocks for parallel execution. Each thread represents a path of execution that can run on a separate core. Parallelism lies the idea of breaking down a task into smaller sub-tasks that can be executed independently and simultaneously. In a multi-core system, each core can handle one or more of these sub-tasks, thus reducing the overall execution time.

public async Task ProcessDataAsync()
{
    // Assign task processing to CPU Core 1
    var task1 = Task.Run(() => HeavyCalculation()); 
    
    // Assign task processing to CPU Core 2
    var task2 = Task.Run(() => AnotherHeavyCalculation()); 
		
    // Wait for both tasks to complete
    await Task.WhenAll(task1, task2);
}        

Important note 1. Task.Run() doesn’t explicitly force the tasks to use different cores, the .NET thread pool and the OS will typically distribute the work across multiple cores if possible. When we call Task.Run(), .NET queues the tasks to the thread pool. The thread pool manages a pool of worker threads. If two tasks are scheduled and there are multiple CPU cores available, the thread pool can assign each task to a different core. This will allow the n tasks to run truly in parallel, taking advantage of the multi-core processor.

Important note 2. The exact distribution of tasks across CPU cores is handled by the operating system’s scheduler and .NET’s thread pool, so while the tasks are likely to run on separate cores, it's not guaranteed that they will always be on different cores. However, the thread pool generally optimizes for parallel execution across available CPU cores.

Once the task completes, it marks itself as completed. At this point, it checks if there are any continuations - if it has a continuation registered (such as code after await), the Task system will schedule this continuation to run. The task does not always continue on the same thread that started it. Once a task finishes, the continuation could run on any available thread, typically one from the ThreadPool. The task system uses a state machine (it is described on the previous post series) under the hood for async methods. It records the state before the await and knows exactly where to resume after the await.

Thus, we don't need to wait for task1 to finish before starting task2. We can run both tasks (or even more) in parallel, speeding up the overall execution time. However, such parallel execution introduces various complexities, including race conditions, deadlocks, and resource contention, which require management of threads.

In the next article, we'll explore how to handle threads and concurrent access to resources effectively.