A bug that vanishes whenever you look for it is maddening. Here is how to debug race conditions: make the failure repeatable, run a race detector, then fix the shared access with a lock or a copy.
Race conditions are one family of bugs within the wider set of debugging techniques, and they reward a methodical approach over guesswork.
| Item | Value |
|---|---|
| Flag that turns on Go’s race detector | -race (for example, go test -race mypkg) |
| Default exit status after a detected race | 66 |
| Default halt_on_error setting | 0 (controls whether the program exits after the first data race) |
| Default history_size setting | 1 |
| Default atexit_sleep_ms setting | 1000 milliseconds |
| Typical output of the loop counter race | 55555 instead of 01234 |
These figures come from the Go data race detector documentation. Defaults can change between releases, so check the source before you rely on them.
Why are race conditions so hard to catch?
A race condition is a bug whose outcome depends on the timing of concurrent work. The same code can pass many runs and then fail on the next, which is why the Go documentation says data races are among the most common and hardest to debug types of bugs in concurrent systems.
A data race is the most concrete form of the problem. It occurs when two goroutines access the same variable concurrently and at least one of the accesses is a write. Other languages use threads instead of goroutines, but the shape of the bug is the same.
The consequences can be severe. The Go documentation shows a data race on a map that can lead to crashes and memory corruption, as described on the same documentation page.
Make the failure easier to trigger
You cannot fix what you cannot reproduce, so start by shrinking the problem. A failure that appears once a week is a poor debugging target, while one that appears every few runs is workable.
- Write down the exact symptom: the wrong value, the crash message or the hang.
- List the shared state involved: variables, maps, files or connections that more than one thread or goroutine touches.
- Put the suspect code in a small test and run it in a loop so the timing window is hit more often.
- Change one thing at a time, such as the amount of parallel work, and note whether the failure rate moves.
Resist the urge to add print statements everywhere. Logging changes timing, and a race that disappears when you add a log line has told you something, but it has not been fixed.
Use a race detector before you read code
Go includes a built-in data race detector. To use it, add the -race flag to the go command, as shown in the Go documentation:
- go test -race mypkg tests a package.
- go run -race mysrc.go runs a source file.
- go build -race mycmd builds a command.
- go install -race mypkg installs a package.
Other ecosystems have equivalents. The Google Testing Blog published a post titled “ThreadSanitizer: Slaughtering Data Races” on June 30, 2014, which is a good starting point if you work in a language that ThreadSanitizer supports.
Whatever the tool, start with your existing tests. A detector turns a bug that fails once in a while into a report that names the conflicting lines every time it observes the race.
How to read a data race report
When the detector finds a race, it prints a report that starts with WARNING: DATA RACE. The report contains stack traces for the conflicting accesses, as well as the stacks where the involved goroutines were created, according to the Go documentation.
If stack traces are unfamiliar, the guide on how to read a stack trace is worth a look first. In a race report, work through it in this order:
- Find the access that is a read or write, and the earlier conflicting access by another goroutine.
- Identify the variable both lines touch. That variable is the shared state to protect.
- Read the creation stacks to see which code started each goroutine.
Tune the detector with GORACE
The GORACE environment variable sets race detector options. The exit status defaults to 66 after a detected race, and halt_on_error defaults to 0, which controls whether the program exits after reporting the first data race. Both are listed in the Go documentation.
Other options are useful in practice:
- log_path writes the report to a file instead of stderr.
- strip_path_prefix removes a prefix from reported file paths so reports are shorter.
- history_size and atexit_sleep_ms adjust how much history is kept and how long the main goroutine waits before exiting.
The history_size option defaults to 1. Raising it can avoid a “failed to restore the stack” error in reports, at the cost of increased memory usage, per the Go documentation. If memory keeps growing for another reason, how to debug a memory leak covers that separate problem.
The atexit_sleep_ms option defaults to 1000 milliseconds of sleep in the main goroutine before exiting, which gives late reports a chance to print, as the Go documentation lists.
Typical races and how to fix them
The Go documentation states that all of the following typical races can be detected with the race detector, and each has a small, well-understood fix. Source: Go documentation.
Race on a loop counter
A closure started inside a loop uses the same loop variable as the loop itself, so its read races with the increment. The program typically prints 55555, not 01234, according to the Go documentation.
The fix is to make a copy of the variable and pass it into the goroutine as a parameter, so each goroutine reads its own local value.
Accidentally shared variable
A common version involves an error variable declared outside a goroutine and then assigned inside it. The goroutine’s write races with a later write in the main goroutine.
The documented fix is to introduce new variables inside the goroutines, using := so each one gets its own err.
Unprotected global variable
Concurrent reads and writes of the same map are not safe. If a registry-style map is touched from several goroutines, protect every read and every write with a mutex, as the Go documentation shows.
Primitive unprotected variable
Data races can happen on variables of primitive types such as bool, int and int64. The documentation warns that even “innocent” data races can lead to hard-to-debug problems caused by non-atomicity of the memory accesses.
What to do when the detector finds nothing
A clean run does not prove your code is safe. The race detector only finds races that happen at runtime, so it cannot find races in code paths that are not executed.
If your tests have incomplete coverage, you may find more races by running a binary built with -race under a realistic workload. That advice comes from the Go documentation.
Try these next steps when a bug still hides:
- Run the race-enabled binary against traffic or data that looks like production.
- Repeat the slow, parallel tests many times rather than once.
- Reduce shared state so fewer variables can race at all.
- Prefer passing data between goroutines over sharing it, where your design allows.
When you build with the -race flag, the go command defines an additional build tag named race. You can use it to exclude code or tests, but treat any exclusion as temporary and record the reason next to it.
Frequently asked questions
Is a race condition the same as a data race?
Show answer
Not quite. A data race is a specific, detectable situation: concurrent access to one variable where at least one access is a write. A race condition is the broader category of timing-dependent bugs, and the Go documentation defines only the data race.
Can the race detector prove my code is race free?
Show answer
No. It reports races that actually happen while the program runs, so code paths that never execute are invisible to it. Broader tests or a realistic workload improve your odds of seeing a race.
Does the race detector have a cost?
Show answer
Yes. The Go documentation’s own examples describe tests that fail under the detector due to timeouts or take too long, and a larger history_size costs more memory. Use it in testing and targeted runs rather than assuming it is free.
Is a race on a simple bool or int safe to ignore?
Show answer
No. The Go documentation says even “innocent” data races on primitive variables can cause hard-to-debug problems because the memory accesses are not atomic. Protect the variable with a mutex or an atomic type instead.
