Go Concurrency Series: Concurrency Patterns(II)

In our last post, we talked about the Worker Pool and Pipeline concurrency patterns, that we can use while designing concurrent applications in Golang. In this article, let’s continue and look at some more concurrency patterns in Golang.

Fan In Pattern

Fan-In is the process of combining multiple results from parallel tasks into a single stream or result. This can be visualized as collecting outputs from various workers and assembling them into a final product. Think of it like what “Reduce” does in Map-Reduce, where it collects data from different workers and aggregates them.

Fan Out Pattern

Fan-Out pattern involves dividing a task into multiple smaller, parallel tasks to be executed concurrently. It's akin to distributing parts of a task across multiple workers in an assembly line, where each worker focuses on a specific part of the job, enhancing overall efficiency and speed.

The above are sample implementations and fan-in fan-out patterns are generally used in conjunction with worker pool and even pipeline patterns.

Handling Errors Across Multiple Goroutines

While working with Go, we generally spin up multiple goroutines and it’s important to handle the lifecycle of the goroutines esp in cases of failure. We’ll look at multiple approaches on how we can coordinate errors across multiple goroutines.

Tomb

The tomb package offers a structured way to manage goroutine lifecycles and handle errors, providing a higher-level abstraction over the native Go concurrency primitives like channels and wait groups. It's particularly useful in applications where clean shutdown and error handling across multiple goroutines are critical. Let's dive into an example of how Tomb helps when we simulate an error in one of the goroutines -

Execution of the above provides the following output -

Context

Contexts are designed to propagate cancellation signals across API boundaries and between goroutines. A context carries deadlines, cancellation signals, and other request-scoped values down the call chain. Since context is part of the standard library, it should be preferred for handling cancellations over tomb. Many standard library functions, especially those involving I/O operations, database queries, and HTTP requests, are designed to accept context.Context as an argument. Let’s see how we can use context for handling cancellations -

Execution of the above provides the following output -

This brings us to the end of this article. Hope you're enjoying the Go Concurrency Series!