Skip to content
javahacks.dev
Concurrency

Fix Deadlocks in Java With Thread Dumps and Lock Ordering

Recognize a Java deadlock in a thread dump, find the two locks and threads involved, and remove the cycle with lock ordering, timeouts or smaller locks.

6 min read
Summarize in AI
Flat illustration of a log with a failing line and a wrench, with the OpenJDK, Spring, Gradle and Apache Maven logos.

The application stops responding, CPU is idle, and requests hang. No exception, no error log. That silence is the signature of a deadlock: two or more threads each hold something the other needs and neither can move. The JVM can tell you which. This post reads a thread dump, finds the cycle and removes it. The tools are covered in jcmd, jstack and jmap in production.

Reproduce the shape

java
class Transfer {
    static final Object A = new Object();
    static final Object B = new Object();

    public static void main(String[] args) {
        new Thread(() -> { synchronized (A) { pause(); synchronized (B) { } } }, "worker-1").start();
        new Thread(() -> { synchronized (B) { pause(); synchronized (A) { } } }, "worker-2").start();
    }

    static void pause() {
        try { Thread.sleep(200); } catch (InterruptedException e) { }
    }
}

worker-1 takes A then wants B; worker-2 takes B then wants A. After the short pause each holds one lock, and both wait forever.

Capture a thread dump

Shell
jcmd <pid> Thread.print

or jstack <pid>, or kill -3 <pid> to print it on the process's standard output. Find the pid with jcmd -l. A dump is a snapshot and costs a brief pause, so it is safe enough in production when taken carefully. Take two a few seconds apart; threads that appear in the same state in both are stuck.

Read the deadlock section

At the end of the dump the JVM prints the cycle:

Text
Found one Java-level deadlock:
=============================
"worker-2":
  waiting to lock monitor 0x00007f3a1c004e80 (object 0x000000076ab62208, a java.lang.Object),
  which is held by "worker-1"

"worker-1":
  waiting to lock monitor 0x00007f3a1c007b00 (object 0x000000076ab62218, a java.lang.Object),
  which is held by "worker-2"

Java stack information for the threads listed above:
===================================================
"worker-2":
	at Transfer.lambda$main$1(Transfer.java:8)
	- waiting to lock <0x000000076ab62208> (a java.lang.Object)
	- locked <0x000000076ab62218> (a java.lang.Object)

The addresses will differ on your run. Read it as a sentence: worker-2 holds lock ...218 and waits for ...208, which worker-1 holds. The stack lines give the source line where each thread is stuck. For java.util.concurrent locks (ReentrantLock), the dump lists "Locked ownable synchronizers", and the report notes the lock owners as well.

You can detect it from inside the application:

java
ThreadMXBean bean = ManagementFactory.getThreadMXBean();
long[] ids = bean.findDeadlockedThreads();      // null when there is none

A watchdog thread that calls this every 30 seconds and logs the thread info turns a silent hang into an alert.

Fix 1: one lock order

If every path acquires A before B, no cycle can form:

java
synchronized (A) { synchronized (B) { /* work */ } }

For locks that are chosen at runtime, such as two accounts in a transfer, order them by a stable key:

java
Account first = from.id() < to.id() ? from : to;
Account second = first == from ? to : from;
synchronized (first) { synchronized (second) { /* transfer */ } }

Fix 2: try with a timeout

java
if (a.tryLock(1, TimeUnit.SECONDS)) {
    try {
        if (b.tryLock(1, TimeUnit.SECONDS)) {
            try { /* work */ } finally { b.unlock(); }
        }
    } finally { a.unlock(); }
}

A thread that cannot get both locks within the time backs off and retries or fails. This uses ReentrantLock, covered in how to use ReentrantLock. The cost is more code and the need to handle the failure.

Fix 3: hold fewer locks for less time

The best fix removes the nested locking. Do not call listeners, callbacks or overridable methods while holding a lock, because you cannot know what they lock. Copy the data you need under the lock, release it and then call out. Prefer immutable data and concurrent collections.

Database deadlocks look different

Two transactions locking rows in opposite orders deadlock inside the database, which kills one and reports an error. That is not a thread dump matter; see locking failures in Hibernate.

Let the JVM detect it from inside

A thread dump is taken by a person after the symptom. The same detector the dump uses is available to code, so a service can notice a deadlock by itself and report it:

java
@Component
class DeadlockWatchdog {

    private static final Logger log = LoggerFactory.getLogger(DeadlockWatchdog.class);
    private final ThreadMXBean threads = ManagementFactory.getThreadMXBean();

    @Scheduled(fixedDelay = 30_000)
    void check() {
        long[] ids = threads.findDeadlockedThreads();
        if (ids == null) {
            return;
        }
        for (ThreadInfo info : threads.getThreadInfo(ids, true, true)) {
            log.error("Deadlocked thread {} waiting for {} held by {}",
                    info.getThreadName(), info.getLockName(), info.getLockOwnerName());
        }
    }
}

findDeadlockedThreads covers both monitors and java.util.concurrent locks. It does a graph search, so keep the interval at 30 seconds or more. An alert on that log line turns a hang discovered by users into one discovered in half a minute, with the lock names and the owners already in the log.

Deadlocks that involve no synchronized keyword

Not every deadlock appears in the deadlock section of a dump. Three shapes are common, and each looks like a hang with threads parked:

ShapeWhat the dump showsFix
Pool starvation: tasks wait for other tasks in the same poolAll pool threads in Future.get or joinUse a separate pool for the inner tasks, or do not block in a task
Class initialisation cycleThreads blocked in <clinit> of two classesRemove the mutual static dependency
A callback made while holding a lockA thread inside a listener, waiting for a lock the caller holdsCall listeners outside the lock
A lock plus a blocking queueOne thread holds a lock and waits on queue.put, the consumer needs the lockRelease the lock before blocking

The first is the most frequent in modern code. A fixed pool of 8 threads, each running a task that submits a subtask to the same pool and waits for it, has no free thread to run the subtasks once all 8 are busy waiting. No lock is involved, and the JVM does not report a deadlock. The dump shows eight identical stacks in get, and that sameness is the clue.

Take several dumps, not one

A single dump shows a moment. A thread that appears in WAITING may be moving on in the next second. Take three dumps ten seconds apart and compare:

Shell
for i in 1 2 3; do jcmd 4242 Thread.print > dump-$i.txt; sleep 10; done
diff <(grep -A3 'pool-1-thread-1' dump-1.txt) <(grep -A3 'pool-1-thread-1' dump-3.txt)

A thread with the same stack in all three is stuck. A thread with a different stack each time is only busy. For applications that use virtual threads, jcmd <pid> Thread.dump_to_file -format=json dump.json records them too, and the plain Thread.print output omits them.

Make the fix a test

A fixed lock order is a convention, and conventions rot. A test that runs two transfers in opposite directions in many iterations catches a regression:

java
@Test
void oppositeTransfersDoNotDeadlock() throws Exception {
    Account a = new Account(1, 1_000);
    Account b = new Account(2, 1_000);
    ExecutorService pool = Executors.newFixedThreadPool(2);

    Future<?> f1 = pool.submit(() -> repeat(10_000, () -> transfer(a, b, 1)));
    Future<?> f2 = pool.submit(() -> repeat(10_000, () -> transfer(b, a, 1)));

    f1.get(10, TimeUnit.SECONDS);
    f2.get(10, TimeUnit.SECONDS);
    pool.shutdownNow();
}

The timeout turns a hang into a failure. Without it the build waits until the CI job is killed. The test is probabilistic, and 10,000 iterations on two threads reproduce the classic lock-order deadlock almost every time.

Before you reach for a timeout

Using tryLock with a timeout makes the program continue after a deadlock, with an error in place of a hang. It treats the symptom, and the cycle remains. Use the timeout as a safety net around code that you cannot restructure, log every time it fires, and still fix the order. A timeout that fires silently hides a design fault for years.

Where this does not apply

A hang with no deadlock section may be a thread waiting on I/O, a pool exhausted by blocking tasks (see rejected executions) or a lost signal. Look at what the threads are doing. The [ThreadMXBean documentation](https://docs.oracle.com/en/java/javase/25/docs/api/java.management/java/lang/management/ThreadMXBean.html) and Java Concurrency in the Java tutorials describe detection and the problem.

Share

Keep reading

Get new posts by email

We send one email when a new post is published. No spam.

Unsubscribe in one click. No spam, ever.