Skip to content
javahacks.dev
Microservices

How to Implement Retries Timeouts and Bulkheads in Java

Build resilient calls in plain Java: bounded timeouts, retries with backoff and jitter, a semaphore bulkhead and how the three combine safely.

6 min read
Summarize in AI
Flat illustration of services connected around a gateway, with the OpenJDK, Docker, Spring Boot and Apache Kafka logos.

Calls between services fail in boring ways: a slow response, a connection that resets, a dependency that is briefly overloaded. A service that does nothing about this turns a minor fault in one dependency into an outage everywhere, because every caller waits, retries and piles on. Three small mechanisms, applied in a particular order, prevent most of that. A timeout stops a call from waiting forever. A retry with backoff and jitter absorbs transient errors. A bulkhead limits how much of your capacity one dependency can consume. This post builds each in plain Java and shows the Resilience4j equivalents, on JDK 25 with Spring Boot 4.

1. Timeouts

The first mechanism is the most important, and the most often missing. A call without a timeout can wait as long as the other side takes, and under failure that is indefinitely. Every thread that waits is a thread not serving other requests, and a pool of 200 threads is gone in seconds when a dependency stops answering.

src/main/java/com/example/client/PriceClient.javajava
HttpClient http = HttpClient.newBuilder()
    .connectTimeout(Duration.ofSeconds(1))
    .build();

HttpRequest request = HttpRequest.newBuilder(URI.create("https://prices.internal/api/price/42"))
    .timeout(Duration.ofSeconds(2))
    .build();

HttpResponse<String> response = http.send(request, HttpResponse.BodyHandlers.ofString());

There are two timeouts and they mean different things. The connect timeout bounds establishing the connection, and should be short (a second or less inside a data centre). The request timeout in the Java client bounds the time until the response is complete. HTTP libraries also have a separate read timeout between bytes, and a total timeout, with different names in each: check what your client's single timeout actually covers. Choose values from measurements: set the timeout a little above the dependency's 99th percentile latency, not at an arbitrary ten seconds.

A timeout is also a budget. If your own endpoint promises an answer within 3 seconds and it calls two dependencies in sequence, their timeouts plus retries must fit inside it. Propagate a deadline down the call chain where you can, rather than giving every call a fresh timeout. Diagnosing timeouts that do not behave is covered in network timeout issues.

2. Retries with backoff and jitter

Some failures disappear on the next attempt: a connection reset, a 503 during a rolling deploy, a leader election. A retry turns those into a slightly slower success. Done carelessly it also multiplies load on a struggling service, which is how retries cause outages.

java
static <T> T retry(Callable<T> call, int maxAttempts, Duration base) throws Exception {
    Exception last = null;
    for (int attempt = 1; attempt <= maxAttempts; attempt++) {
        try {
            return call.call();
        } catch (IOException | TransientException e) {
            last = e;
            if (attempt == maxAttempts) break;
            long cap = base.toMillis() * (1L << (attempt - 1));
            long sleep = ThreadLocalRandom.current().nextLong(cap + 1);   // full jitter
            Thread.sleep(sleep);
        }
    }
    throw last;
}

Three rules make a retry safe.

Retry only what is safe to repeat. A GET is; a PUT that replaces a resource is; a POST that creates an order is not, unless the API supports an idempotency key and you send one, as in idempotency keys. Retrying a payment without one charges the customer twice. Retry only errors that are transient: connection failures, timeouts, 502, 503, 504, and 429 honouring Retry-After. Never retry a 400 or a 404; the second answer will be the same.

Back off exponentially, so each wait is longer than the last, and add jitter, a random element, so that a hundred clients that failed together do not retry together. The "full jitter" form above picks a random delay between zero and the exponential cap, and spreads the load evenly. Without jitter, retries arrive in synchronised waves that hit the recovering service just as it comes up.

Limit attempts, usually two or three, and respect the budget from the timeout section: three attempts of two seconds each take six seconds plus waits. Blocking with Thread.sleep ties up a thread, which is cheap on a virtual thread and expensive on a platform thread pool.

3. Bulkhead

A bulkhead isolates failures the way a ship's compartments do. If calls to the recommendations service slow down, they must not consume every thread that the checkout needs. Cap the concurrency per dependency:

java
Semaphore recommendations = new Semaphore(20);

List<Item> recommend(Customer c) {
    if (!recommendations.tryAcquire()) {
        return List.of();                       // shed load: return a fallback now
    }
    try {
        return client.recommend(c);
    } finally {
        recommendations.release();
    }
}

With at most twenty concurrent calls, a slow dependency can hold twenty threads and no more. The twenty-first caller is turned away immediately, with a fallback, instead of queueing behind a dependency that is not answering. The fallback is a design decision: an empty list for a nice-to-have, a cached value for something stale-tolerant, an error for something essential. The semaphore pattern is explained in limiting access with semaphores; a separate thread pool per dependency is the heavier alternative, and a full pool then throws, which is the topic of RejectedExecutionException.

Putting them together

The order of wrapping matters. From the outside in: bulkhead, then retry, then the timeout on each attempt. The bulkhead guards the dependency as a whole, the retry decides whether to try again, and each attempt has its own timeout. Do it the other way and a retry loop is inside the timeout, so the timeout cuts the retries short and wastes them.

Resilience4j provides these as composable decorators:

java
CheckedFunction0<Price> call = Decorators.ofCheckedSupplier(() -> client.price(42))
    .withBulkhead(bulkhead)
    .withRetry(retry)
    .withCircuitBreaker(breaker)
    .decorate();

The library adds metrics and event listeners, and the circuit breaker, which stops calling a dependency that is clearly down and lets it recover. Configuration through Spring Boot properties and the annotations are in Resilience4j bulkheads and rate limiters. The Azure architecture pages on the retry pattern and the circuit breaker pattern describe the ideas independently of any library.

What can go wrong

Retries at every layer multiply. If the gateway retries three times, the service retries three times and the client library retries three times, one failed request becomes twenty-seven attempts. Retry at one layer, usually the one closest to the user or the one that knows whether the operation is idempotent, and make the others fail fast. A retry storm during an incident makes a partial outage total, so monitor the retry rate as a metric and alert on it.

A timeout shorter than the dependency's normal latency makes every call fail, and a retry on top doubles the load while achieving nothing. Set timeouts from percentiles and review them when latency changes. Queues without bounds hide trouble: a bulkhead that queues instead of rejecting turns overload into latency, then into timeouts. Prefer to reject quickly. Test the behaviour, not only the code, by injecting latency and failures in a test environment, and verify what the system does when a dependency is down for five minutes.

Where this does not apply

For asynchronous messaging, the equivalents are consumer retries with a dead letter queue, which have their own rules. For calls inside a single process, timeouts and bulkheads still apply to anything that can block, such as a lock or a pool, but retries rarely do. A library that already provides these, such as the gRPC client's retry policy, should be configured instead of wrapped.

Taggedhow to
Share

Keep reading

Get new posts by email

We send one email when a new post is published. No spam.

Unsubscribe in one click. No spam, ever.