Worker Pools: Fan-Out and Fan-In

Spawning one goroutine per job is Go’s party trick, and for a few hundred jobs it is genuinely fine. But “a goroutine per job” stops being a plan the moment the number of jobs is large or unbounded. Ten thousand jobs means ten thousand goroutines all contending for the same database connection, the same rate-limited API, the same finite pool of file handles. You do not want ten thousand things happening at once. You want some fixed number happening at once, with the rest waiting their turn.

That fixed number is the whole point of a worker pool. You start N worker goroutines, hand them a shared channel of jobs, and let them pull work off it until it runs out. N is the ceiling on concurrency, chosen by you, independent of how many jobs there are. This is the pattern you reach for when the reason to go concurrent is throughput, not unbounded parallelism.

Fanning out: N workers, one jobs channel

A worker is a goroutine that ranges over a shared jobs channel and sends each result onto a shared results channel:

// worker pulls jobs off the shared channel until it is closed and drained,
// squaring each and sending the result on.
func worker(id int, jobs <-chan int, results chan<- int, wg *sync.WaitGroup) {
	defer wg.Done()
	for j := range jobs {
		fmt.Printf("worker %d handled job %d\n", id, j)
		results <- j * j
	}
}

The magic is that every worker ranges over the same channel. When multiple goroutines receive from one channel, the runtime hands each value to exactly one of them; a job is never delivered twice, and no two workers ever collide over the same job. The channel is the work queue and the dispatcher at once. Ranging over it means each worker keeps taking jobs until the channel is closed and empty, then falls out of its loop and calls wg.Done(). Notice the parameter types: jobs is receive-only (<-chan int) and results is send-only (chan<- int), which is the compiler enforcing that a worker consumes jobs and produces results, not the reverse.

Now the coordination in main, which is fussier than it looks and worth reading slowly:

func main() {
	const numJobs = 9
	jobs := make(chan int, numJobs)
	results := make(chan int, numJobs)

	// Three workers, no matter how many jobs. This is the bound.
	var wg sync.WaitGroup
	for w := 1; w <= 3; w++ {
		wg.Add(1)
		go worker(w, jobs, results, &wg)
	}

	for j := 1; j <= numJobs; j++ {
		jobs <- j
	}
	close(jobs) // no more work; each worker's range ends when the buffer drains

	// Close results once every worker has returned.
	go func() {
		wg.Wait()
		close(results)
	}()

	sum := 0
	for r := range results {
		sum += r
	}
	fmt.Println("sum of squares:", sum)
}

worker 3 handled job 3
worker 3 handled job 4
worker 3 handled job 5
worker 3 handled job 6
worker 3 handled job 7
worker 3 handled job 8
worker 3 handled job 9
worker 2 handled job 2
worker 1 handled job 1
sum of squares: 285

Three things are choreographed here, and each solves a specific problem.

close(jobs) signals “no more work.” Once you have sent every job, closing the channel tells the workers, via their range loop ending, that nothing more is coming. Without it, the workers block forever on an empty channel waiting for a job that never arrives. Closing is how you say “done sending” to a receiver.

The WaitGroup knows when every worker has finished. Each worker calls wg.Done() as it exits; wg.Wait() blocks until all three have. The WaitGroup was covered a few chapters back; here it answers exactly one question, “have all the workers stopped?”, so you know when it is safe to close results.

close(results) has to happen in its own goroutine, and only after wg.Wait(). This is the subtle bit. You cannot close results before the workers are done, or a worker will send on a closed channel and panic. You cannot call wg.Wait() on the main goroutine before the for r := range results loop, because the workers are blocked trying to send into results while main is blocked waiting for them, a deadlock. So the closer runs concurrently: wg.Wait() then close(results) in a goroutine, while main drains results on the other side. When the last worker finishes and results closes, main’s range loop ends.

The total is fixed; the assignment is not

Run that program again and the last three lines shuffle:

worker 1 handled job 1
worker 1 handled job 4
worker 1 handled job 5
worker 1 handled job 6
worker 1 handled job 7
worker 1 handled job 8
worker 1 handled job 9
worker 3 handled job 3
worker 2 handled job 2
sum of squares: 285

Two truths sit side by side in that output, and holding both is the key to reasoning about pools.

Which worker handles which job is nondeterministic. Across runs the assignment changes completely: in the first run worker 3 grabbed a long burst; in the second, worker 1 did. Because jobs is a buffered channel loaded up front, whichever worker the scheduler runs first can snatch a run of jobs before its peers even start. A pool does not promise fair or even distribution, and it is not round-robin. It promises only that at most N workers run at once and that every job is handled exactly once by exactly one of them.

The sum is always 285. The squares of 1 through 9 add to 285 no matter who computed which. This is the property to lean on: the aggregate is deterministic even though the route to it is not. So when you verify a pool, assert on the total, the count, the set of results collected, never on the order they arrive or on which worker did what. Counted claims live on the aggregate; the ordering is honest noise. (Run under go run -race, the shared jobs and results channels come back clean, because channel operations are synchronized by construction.)

Fanning in: merging channels back into one

Fan-out splits a stream across workers. Fan-in does the reverse: it merges several channels into one, so a single range downstream can drain them all. The tool is again a WaitGroup, this time counting the merge goroutines.

// merge fans several channels into one: a goroutine per input forwards to a
// shared output, and the output closes once all of them have drained.
func merge(cs ...<-chan int) <-chan int {
	out := make(chan int)
	var wg sync.WaitGroup
	wg.Add(len(cs))
	for _, c := range cs {
		go func() {
			defer wg.Done()
			for v := range c {
				out <- v
			}
		}()
	}
	go func() {
		wg.Wait()
		close(out)
	}()
	return out
}

One forwarding goroutine per input channel copies every value onto the shared out. A separate goroutine waits for all the forwarders to finish, then closes out, the identical close-after-Wait dance from the pool, because the problem is identical: close the shared output exactly once, only after every sender is done.

func main() {
	a := producer(1, 2, 3)
	b := producer(10, 20, 30)
	c := producer(100, 200, 300)

	var got []int
	for v := range merge(a, b, c) {
		got = append(got, v)
	}
	sort.Ints(got) // sort only so the printed line is stable; arrival order is not
	fmt.Println("merged:", got)
	fmt.Println("count:", len(got))
}

merged: [1 2 3 10 20 30 100 200 300]
count: 9

The count is a rock-solid 9: three inputs of three values, none lost. The order, though, is whatever interleaving the scheduler produced, which is why the code sorts before printing, purely to get a stable line to show you. The one loop-body detail worth flagging: the goroutines close over the loop variable c directly, which is correct here only because Go 1.22 gave each loop iteration its own copy of the variable. Written against an older Go, every goroutine would have raced on a single shared c and forwarded the wrong channel. On 1.26 it does what it reads like.

Choosing the pool size

The whole reason to use a pool is to pick N deliberately, so it is worth a word on how. The right N is set by the resource you are protecting, not by the number of jobs. If each worker holds a database connection and the pool has 20 connections, N is around 20; more workers just queue on the connection pool. If the work is CPU-bound, runtime.NumCPU() is a sensible ceiling, because more runnable goroutines than cores only adds scheduling overhead. If the work is network-bound and waits on a rate-limited API, N is whatever the rate limit allows. The failure mode of guessing high is the exact thing the pool exists to prevent: too many workers contending for a scarce resource, which is why “one goroutine per job” is the anti-pattern here. Start from the bottleneck and size the pool to it.

Two ends of the pool also want thought in real code. This example squares numbers and never fails, but real jobs return errors, and a worker needs somewhere to put them, usually a second results-style channel, or the errgroup we reach in the next chapter. And if a job can hang, a worker blocked on it takes a slot out of the pool until it returns, so a pool over unreliable work wants a per-job timeout (a context deadline) so one stuck job cannot starve the whole pool down to N minus one.

Bounding concurrency with a semaphore

A worker pool bounds concurrency by having a fixed set of goroutines pull from a queue. But sometimes you genuinely want one goroutine per job — each job kept on its own goroutine for clarity, or because a job carries per-request state you would rather not multiplex — and you only want a cap on how many run at once. The tool for that is a counting semaphore, and in Go it is just a buffered channel used for its capacity.

The idiom is three lines of setup and two operations. Make a channel with buffer N. To acquire a slot, send an empty struct into it; the send blocks once N values are already buffered, which is exactly the cap doing its job. To release, receive one back out.

const limit = 3
sem := make(chan struct{}, limit)

var wg sync.WaitGroup
for job := range 20 {
	wg.Add(1)
	sem <- struct{}{} // acquire: blocks when N are already in flight
	go func() {
		defer wg.Done()
		defer func() { <-sem }() // release the slot

		// ... do the job; at most `limit` of these run at once ...
	}()
}
wg.Wait()

peak concurrency: 3 (cap 3 )

Twenty jobs, each on its own goroutine, but the sem <- struct{}{} before the go statement means the loop blocks whenever three are already running — it cannot launch a fourth until one of the three finishes and its deferred <-sem frees a slot. The verification tracks the number in flight with an atomic counter and records the high-water mark: across all twenty jobs the peak was exactly 3, never 4. The cap holds. struct{} is the conventional element type because it occupies zero bytes — you only ever care about the channel’s count, never the values, so there is no reason to move real data.

Two details make this correct. The acquire goes before go, not inside the goroutine — put it inside and all twenty goroutines launch immediately and then queue on the send, which defeats the point of limiting how many exist. And the release is a defer, so a slot is returned even if the job panics or returns early. This differs from a worker pool in a way worth holding onto: a pool creates N goroutines total and feeds them many jobs; a semaphore creates one goroutine per job and lets only N proceed. Reach for the pool when spawning is the cost you are avoiding; reach for the semaphore when you want a goroutine per job but a ceiling on simultaneity.

For the weighted case — where jobs are not equal and one heavy job should count as, say, three units against the limit — golang.org/x/sync/semaphore provides a Weighted semaphore with Acquire(ctx, n)/Release(n) that also respects context cancellation while waiting. The buffered-channel version above is the right default; reach for the package when your slots have unequal weights.

Final thoughts

A worker pool bounds concurrency: N workers all ranging over one shared jobs channel, so the runtime dispatches each job to exactly one of them and at most N run at once. The coordination is a small fixed recipe, close(jobs) to end the workers’ range, a WaitGroup to learn when they have all stopped, and a close(results) that must run in its own goroutine after wg.Wait() so main can drain the other side. Reason about the aggregate, which is deterministic; ignore which worker did what, which is not. Fan-in is the mirror image, merging channels with a goroutine apiece and the same close-after-Wait, and together fan-out and fan-in are how you build throughput out of a bounded, orderly set of goroutines. But every one of these patterns rests on goroutines actually finishing, and next we look at what happens when they do not.

Next: the goroutine that never returns — how a blocked goroutine leaks, why the race detector stays silent, and how to prove it with a goroutine count.