Skip to content
javahacks.dev
Testing

How to Test Concurrent Code and Find Race Conditions

Test concurrent Java code and find race conditions: stress tests with many threads, latches, repeated runs, jcstress and thread-safety design.

6 min read
Summarize in AI
Flat illustration of a test report with one failing test, with the OpenJDK, GitHub Actions and Selenium logos.

A race condition is a bug whose appearance depends on timing. A test that passes ten thousand times can fail on the next run. Testing for these bugs means increasing the chance of the bad interleaving and checking results that must hold. Examples use JUnit 5 and JDK 25.

A broken counter

java
class Counter {
    private int value;
    void increment() { value++; }       // not atomic
    int get() { return value; }
}

value++ is read, add, write. Two threads can read the same value and both write value + 1, losing an update.

A stress test

java
@RepeatedTest(20)
void incrementIsThreadSafe() throws Exception {
    int threads = 16, perThread = 10_000;
    Counter counter = new Counter();
    CountDownLatch ready = new CountDownLatch(threads);
    CountDownLatch start = new CountDownLatch(1);
    ExecutorService pool = Executors.newFixedThreadPool(threads);

    List<Future<?>> futures = new ArrayList<>();
    for (int i = 0; i < threads; i++) {
        futures.add(pool.submit(() -> {
            ready.countDown();
            start.await();                           // all threads start together
            for (int n = 0; n < perThread; n++) counter.increment();
            return null;
        }));
    }
    ready.await();
    start.countDown();
    for (Future<?> f : futures) f.get(10, TimeUnit.SECONDS);   // surface exceptions
    pool.shutdown();

    assertEquals(threads * perThread, counter.get());
}

The start latch releases all threads together to maximise contention. Futures' get rethrows exceptions from workers, which would otherwise vanish. The test checks an invariant: the total must equal the number of increments. With the broken counter it fails almost every time; with AtomicInteger or a lock it passes. @RepeatedTest raises the chance of catching rarer races. Use more threads than cores for more interleavings.

Assert invariants, not order

Good concurrent tests check properties that hold regardless of timing:

  • totals and balances after many operations,
  • no element lost or duplicated in a queue,
  • a value never goes below zero,
  • every task completes exactly once.

Do not assert on a specific order of events unless that order is guaranteed.

Deadlock detection

A test that hangs is a failing test. Always use timeouts:

java
@Test
@Timeout(10)
void transferDoesNotDeadlock() {
    // two threads transferring in opposite directions
}

Use assertTimeoutPreemptively for code that may block forever, and on timeout capture a thread dump, see deadlock diagnosis.

jcstress

The OpenJDK Java Concurrency Stress tool runs small tests that explore memory model outcomes:

java
@JCStressTest
@Outcome(id = "1, 1", expect = ACCEPTABLE, desc = "both see the other's write")
@Outcome(id = "0, 0", expect = ACCEPTABLE_INTERESTING, desc = "neither sees it")
@State
public class VisibilityTest {
    int x, y;
    @Actor public void actor1(II_Result r) { x = 1; r.r1 = y; }
    @Actor public void actor2(II_Result r) { y = 1; r.r2 = x; }
}

jcstress executes this billions of times across threads and reports which outcomes occurred. It is for library authors and for checking claims about volatile and ordering, see the Java memory model. Most application code does not need it.

Tools that help

  • Java Flight Recorder and thread dumps for contention and blocked threads.
  • IntelliJ IDEA concurrency inspections and static analysis such as Error Prone's @GuardedBy checks.
  • Lincheck (Kotlin and Java) checks linearizability of concurrent data structures by exploring interleavings.
  • Chaos-style delays: inject small random sleeps in test builds to widen race windows.

Design out the problem

Tests find some bugs; design prevents most:

  1. Prefer immutable objects, see the final keyword.
  2. Confine mutable state to one thread or one actor.
  3. Use java.util.concurrent classes, atomics and ConcurrentHashMap correctly, see atomics and LongAdder.
  4. Document which lock guards which field.
  5. Keep critical sections small.

Test pyramid placement

Keep these tests few and focused on the thread-safe components. Run the repeated stress tests in a separate CI stage, since they are slow. A failing concurrency test should not be retried until green; treat it as a bug report, see async testing.

Watch the counter lose updates

The broken counter in the section above is worth running instead of trusting. This harness starts eight threads, holds them at a gate until all are ready so they begin together, and has each call the operation 100,000 times. Five runs of a plain int counter and one run of an AtomicInteger produced this output on JDK 17 with ten cores.

Race.javajava
static int hammer(Runnable op, int threads, int perThread) throws Exception {
    ExecutorService pool = Executors.newFixedThreadPool(threads);
    CountDownLatch ready = new CountDownLatch(threads), go = new CountDownLatch(1);
    List<Future<?>> fs = new ArrayList<>();
    for (int t = 0; t < threads; t++) fs.add(pool.submit(() -> {
        ready.countDown();
        try { go.await(); } catch (InterruptedException e) { return; }
        for (int i = 0; i < perThread; i++) op.run();
    }));
    ready.await();
    go.countDown();
    for (Future<?> f : fs) f.get();           // rethrows anything a worker threw
    pool.shutdown();
    return threads * perThread;
}
Text
plain  run 1: expected 800,000 got 206,133 lost 593,867
plain  run 2: expected 800,000 got 426,458 lost 373,542
plain  run 3: expected 800,000 got 594,710 lost 205,290
plain  run 4: expected 800,000 got 503,728 lost 296,272
plain  run 5: expected 800,000 got 542,405 lost 257,595
atomic       : expected 800,000 got 800,000

Every plain run lost between a quarter and three quarters of the increments, and no two runs lost the same amount. That is what a race looks like: wrong, different each time, and silent. A real service rarely loses 60 percent, because its threads do other work between updates, and the same defect there loses one update in a million and passes every review. The harness works because it removes the other work and makes the window as wide as possible.

Two details make the harness fair. The go latch releases all the threads at the same instant, so they overlap instead of finishing one after another as threads start. And f.get() for each future matters: an exception in a worker thread disappears otherwise, and the test would pass with a thread that crashed in its first iteration.

Make a failing test reproducible

A stress test that fails once in fifty runs is useful only if you can run it fifty times cheaply. Use JUnit's repetition and keep the body short enough to run a few thousand times in seconds.

java
@RepeatedTest(200)
void counterNeverLosesAnUpdate() throws Exception {
    SafeCounter counter = new SafeCounter();
    int expected = hammer(counter::inc, 8, 5_000);
    assertEquals(expected, counter.n.get());
}

Run it on the build server as well as locally, with more threads than cores. Oversubscribing the CPU forces the scheduler to preempt threads in the middle of a critical section, which is when most races show. A test that passes on an eight core laptop with eight threads can fail on a two core runner with sixteen.

Detect what a test cannot assert

Some races never change a final result but still corrupt visibility: a flag written by one thread and never seen by another, or a half-constructed object published without a safe handoff. A total cannot catch those. The tool for that class is jcstress, the OpenJDK harness that runs an actor pair millions of times and tallies the outcomes the memory model permits. Use it for the small number of lock-free structures you write yourself, and use java.util.concurrent classes everywhere else, since their authors have already used it.

Thread Sanitizer is the other option for native code, and -XX:+UnlockDiagnosticVMOptions offers little for pure Java. For Java, the practical detectors are a code review against the memory model rules, SpotBugs rules such as IS2_INCONSISTENT_SYNC, and Error Prone checks like GuardedBy, which turn a documented locking rule into a compile error.

java
class Account {
    private final Object lock = new Object();
    @GuardedBy("lock") private long balance;

    void deposit(long amount) {
        synchronized (lock) { balance += amount; }
    }
}

The annotation costs nothing at run time. With Error Prone enabled it fails the build when a method touches balance without holding lock, which finds the bug that the stress test needs luck to see.

Know what the green result proves

A passing stress test shows that the interleavings it produced did not break the invariant. It does not prove the code is correct. Raise confidence by varying the conditions: a different thread count, a different JIT state by running the test once cold and once after a warm-up loop, and different hardware in CI. When the code is a candidate for a library, say so in the review and ask for a design that does not need the test to be lucky: immutable data, confinement to one thread, or a standard concurrent collection.

When this does not apply

Single-threaded code needs none of this. References: jcstress and the Lincheck project.

Taggedhow to
Share

Keep reading

Get new posts by email

We send one email when a new post is published. No spam.

Unsubscribe in one click. No spam, ever.