← All posts

Goroutine Leaks and what a forced leak teaches you

A forced leak is a good way to learn what a stuck goroutine looks like and how a runtime surfaces it. This demo is a compact lesson in verifying lifetime, reading stack signals, and designing concurrency with explicit exits.

Goroutine leaks look small when they start. One blocked send. One receiver that never exits. One background worker with no shutdown path. In Go, those mistakes hide well because goroutines are cheap. The cost shows up later, when work piles up, memory stays live, and shutdown gets harder to reason about.

Goroutine Leaks matters because it turns a vague class of bugs into something you can inspect. The point is not the leak itself. The point is the feedback loop. If your runtime and your code give you a clear signal when a goroutine has no path to completion, you fix the bug while the context is still fresh.

For practitioners, this is about control over concurrency. You want every goroutine to have a job, a lifetime, and an exit condition. When any of those are missing, observability has to do the work your design did not. A small demo of a forced leak is useful because it strips the issue down to first principles.

What a goroutine leak is in practice

A goroutine leak is a goroutine that stays alive after its work should have ended. It is not a memory leak in the strict sense, but it often keeps memory reachable. Stack frames, channel references, timers, contexts, sockets, and captured heap objects stay around because the goroutine is still parked somewhere.

Common shapes include:

  • A goroutine blocked on receive from a channel nobody will close.
  • A goroutine blocked on send to a channel nobody will read.
  • A worker loop with no context.Context or stop signal.
  • A time.Ticker or time.Timer created and never stopped.
  • An HTTP call or RPC without a deadline, waiting forever on I/O.
  • A fan-out pattern where one branch exits early and leaves sibling goroutines running.

The danger is not the raw count alone. You need to read count together with state. A service with many idle goroutines is not always broken. A service with a slowly rising count after each request, deploy, or batch job deserves attention. The leak pattern matters more than the headline number.

This is why a deliberate leak demo helps. You get a known-bad case. Then you inspect what the runtime reports, how the blocked state appears, and which code path failed to close the loop.

What to inspect in the demo

Start with the lifecycle of the leaked goroutine. Ask three questions.

  1. What started it.
  2. What work it is waiting on.
  3. What event would let it exit.

In a healthy design, the third answer is explicit. The goroutine returns when a context is done, when an input channel closes, when a bounded queue drains, or when a shutdown path runs. In a leak, the exit event is missing, unreachable, or owned by code that never runs.

For a forced leak, the blocked state is often one of the standard runtime wait reasons. In practical terms, you look for a goroutine stuck in channel send, channel receive, select, network poll, or sleep tied to a forgotten timer path. The useful signal is the repeatability. If each interaction creates one more stuck goroutine and none leave, you have a leak pattern, not a temporary burst.

When you inspect a goroutine dump, look for:

  • Identical stacks repeated many times.
  • Frames stuck in your package, not only runtime internals.
  • Goroutines created from request handlers or loop bodies.
  • Missing context cancellation or missing channel closure near the top of the stack.
  • Wait states tied to unbuffered channels or unbounded retries.

A good demo makes those signatures easy to see. You do not need a production system to learn the habit. The same reading method applies later under load.

How to verify the claim yourself

The right way to verify a goroutine leak is to reproduce it, then observe both creation and non-termination. You are not trying to prove intent. You are trying to show that a goroutine remains live after the useful work ends.

A practical verification loop looks like this:

  1. Trigger the code path once.
  2. Record goroutine count and, if available, a goroutine profile or dump.
  3. Trigger the same path again.
  4. Compare whether the same blocked stack appears again, with one more instance.
  5. Stop new work and see whether the count returns to baseline.

If the count never returns, inspect stacks. Count alone is weak evidence. Stack repetition is stronger. A blocked goroutine with the same stack frame after the triggering work has finished is the core signal.

In Go, you would usually pair several built-in views:

  • runtime.NumGoroutine() for a quick count trend.
  • pprof goroutine profiles for grouped stack traces.
  • Block and mutex profiles when waiting or contention is part of the path.
  • Heap profiles to see what leaked goroutines keep reachable.
  • Execution traces when scheduler behavior matters.

The demo’s value sits in the phrase “watch Go 1.27 find it.” For an engineer, that means you should look for what the runtime surfaces without custom instrumentation, and how quickly it points you toward the stuck path. The claim is testable. Trigger the intentional leak. Observe whether the runtime exposes a clear, stable signature. Then remove the leak and confirm the signature disappears.

That before-and-after check matters. It tells you whether you found the bug or only changed timing.

Design choices that prevent leaks

Leak prevention starts before profiling. It starts with ownership.

Every goroutine should have:

  • A clear parent operation.
  • A bounded lifetime.
  • A cancellation path.
  • A documented owner for closing channels.

The easiest pattern to reason about is context.Context flowing through every operation that blocks. If a goroutine waits on I/O, a queue, or a long loop, it should also wait on ctx.Done(). If it writes to a channel, think about what happens when the receiver exits first. If it reads from a channel, decide who closes it and when.

Useful habits include:

  • Prefer structured worker lifecycles over fire-and-forget goroutines.
  • Use errgroup or equivalent patterns to tie sibling goroutines to one cancellation domain.
  • Stop tickers and timers in all exit paths.
  • Bound retries and backoff loops.
  • Close response bodies and release resources even on error paths.
  • Make shutdown logic part of tests, not only part of main.

Code review should ask a blunt question for every new goroutine: how does this end? If the answer is fuzzy, the implementation is not ready.

Tests help when they focus on lifetime, not only output. A good concurrency test runs the operation, cancels the context, waits for goroutines to exit, and fails if they linger. This is where small demos pay off. They teach the exact failure shape your tests should catch.

Where this class of system goes wrong

Teams often look for leaks only in memory charts. By then, the symptom is late and noisy. Goroutine leaks often show up first as slow shutdowns, hanging requests, rising file descriptor use, or background activity after work should have quiesced.

Another common mistake is to treat goroutine count as a service health score. Count is useful, but context matters. A bursty worker pool, long-lived subscribers, and idle HTTP keepalive loops all add goroutines for valid reasons. The question is whether they drain when demand drops or shutdown begins.

A third failure mode is partial cancellation. One part of a pipeline respects context. Another part blocks on a send forever. The caller times out, but the internal goroutine stays parked. These are hard bugs because the user-facing symptom goes away while the leaked work stays behind.

You also see leaks around fan-out and fan-in. One branch returns early on error. Another branch is still trying to send results. If the collector has exited, those senders park forever. Buffered channels reduce the odds for small bursts, but they do not fix ownership or shutdown.

Finally, observability often arrives too late. Services expose latency and error metrics, but no goroutine profiles, no block profiles, and no controlled way to capture state during a leak event. When the bug appears, you have a count graph and little else. A small demo is useful because it sets the standard for what good evidence looks like.

What to watch next

The next step is to treat goroutine lifetime as a design property. Add cancellation to blocking paths. Capture goroutine profiles during tests and during controlled load. Watch for repeated blocked stacks, not only rising counts.

If a forced leak is easy to see in a demo, your production systems should expose enough state to spot the same pattern early. That is the practical lesson behind Goroutine Leaks.