Goroutine Leaks
A goroutine leak is a goroutine that is blocked for good: it waits on a
channel, a mutex or a WaitGroup that nothing will ever release, so it never ends and its memory is never
given back. Go 1.27 makes the goroutine leak profile generally available (it was an experiment in Go 1.26):
pprof.Lookup("goroutineleak"), or /debug/pprof/goroutineleak with
net/http/pprof. The runtime finds leaks during a garbage-collection pass: a goroutine blocked on
something no goroutine that can still run could ever reach can never wake up.
The leaks on this page are real, and they do not run here. Each
button asks a sandboxed Go process at leaks.pigfox.com to run the code shown. A leaked goroutine never
ends, so the sandbox replaces its process on a fixed schedule and every
leak starts again from zero. Nothing you click is stored.
Connecting to the sandbox…
The sandbox is not answering
This page cannot reach the sandbox at leaks.pigfox.com right now, so there is nothing live to
show: no buttons, no profile. It will keep trying while this tab is open. What it would show is described
below: five common leaks, each beside its fix, one leak the profile cannot see, and two views of the same
process, the old goroutine dump and the Go 1.27 leak profile.
The sandbox, live
- Goroutines
- —
- Heap
- —
- Leak check
- —
- Leaks started
- —
- Next restart in
- —
Each read of the profile here runs the leak check twice, because one pass can miss a few leaks.
Goroutine dump (what you had before)
Every goroutine that is waiting, grouped by where it waits. The healthy pool and the blind spot look just like the leaks here.
- Waiting for the sandbox.
Leak profile (Go 1.27)
Only the goroutines the runtime proved can never wake. Pick a group to see the line it is stuck on.
- Waiting for the sandbox.
Five leaks, their fixes, and a blind spot
How it works
How the profile finds a leak. The garbage collector normally starts from every goroutine. For the leak check it starts only from goroutines that can run. A blocked goroutine counts as live only if the channel, mutex or WaitGroup it waits on can be reached from one of those, or from a goroutine they could wake. Whatever is left is blocked on something nothing can ever release, and the profile reports it, with the line it is stuck on.
Why the old dump was not enough. A goroutine dump lists every
goroutine and what it waits on. A worker parked on an empty queue and a goroutine that will never wake show
the same state, such as chan receive, so telling them apart meant reading every stack. The
sandbox keeps a small pool of idle workers running for exactly this reason: they sit in the dump beside the
leaks and never appear in the leak profile.
The blind spot. The leak profile cannot report every leak, and the Go team says so: it finds a large class of them. Because it works from reachability, it misses a goroutine blocked on something still reachable from a global variable, or from a goroutine that can still run. The blind-spot card waits on a package-level channel that nothing ever sends on. It is stuck for good, but the channel is reachable from a global, so the runtime cannot tell it from a healthy waiter. It shows up in the goroutine dump and never in the leak profile. Goroutines blocked on file or network I/O or in a system call are never reported either.
The sandbox. A leaked goroutine never ends, so the only way to give its memory back is to end the process. The sandbox caps how many goroutines can be leaked before its next restart, limits how often one address can start runs, and reads the profile at most every few seconds for everyone; this page reads it every 4 seconds while the tab is visible. The figures above are the sandbox's own: its goroutine count, its heap, how long the leak check took, and how many leaks have been started since it last restarted.