Member-only story
Go Concurrency Patterns: Fan-Out and Fan-In
A pattern that distributes workloads across workers and leverages parallelism in multicore CPUs.
In our previous articles, we explored the Generator and Pipeline patterns, which are ideal for scenarios where a single consumer processes a stream of data. These patterns are powerful, but they can be limited in situations where you want to fully leverage the capabilities of modern multi-core processors or need to handle I/O-bound tasks more efficiently. To achieve this, we can extend our approach to distribute workloads across multiple consumers. This is where the Fan-Out and Fan-In concurrency patterns come into play.
In this article, we’ll dive into these two essential patterns. The Fan-Out pattern allows you to parallelize tasks by distributing work across multiple goroutines, while the Fan-In pattern aggregates the results from these parallel tasks back into a single channel. Together, these patterns enable you to maximize concurrency in your Go applications, improving both performance and scalability.
