Skip to content
javahacks.dev
Microservices

Fix Feign and RestClient Timeouts Between Spring Services

Fix Read timed out errors between Spring services by setting connect and read timeouts on RestClient and Feign, sizing pools and finding the slow side.

7 min read
Summarize in AI
Flat illustration of services connected around a gateway, with the Spring, Spring Boot, Apache Kafka, Istio and Redis logos.

Service A calls service B, B is slow for a moment, and A throws one of these:

Text
org.springframework.web.client.ResourceAccessException: I/O error on GET request for "http://inventory/stock/A-100": Read timed out
Text
feign.RetryableException: Read timed out executing GET http://inventory/stock/A-100

Both mean the same thing: A connected to B, sent the request and waited for the response longer than A's read timeout, so A gave up. A timeout is not an error in A. It is A protecting itself from waiting forever on B, and the real questions are whether the timeout was set deliberately, whether its value fits the latency budget of the whole request, and why B was slow. The default, in some clients, is to wait without limit, which converts a slow dependency into a hung thread pool and then into an outage of A. This post sets timeouts explicitly in RestClient and Feign, derives values from a budget, finds why B is slow, and covers retries and circuit breakers without making things worse. Examples use Spring Boot 4, Spring Framework 7 and Spring Cloud OpenFeign.

Set timeouts explicitly

There are two timeouts and they mean different things. The connect timeout limits how long to wait for the TCP connection to B to be established; it fails fast when B is down or unreachable. The read timeout limits how long to wait for response data once the request is sent; it fires when B is up and slow. Set both, always. In Spring Boot 4, global defaults for the HTTP clients are properties:

src/main/resources/application.yamlYAML
spring:
  http:
    client:
      connect-timeout: 1s
      read-timeout: 3s

For a specific RestClient, set them on the request factory so different dependencies get different values:

src/main/java/dev/javahacks/clients/InventoryClient.javajava
@Configuration
class InventoryClientConfig {

    @Bean
    RestClient inventoryRestClient(RestClient.Builder builder) {
        var settings = ClientHttpRequestFactorySettings.defaults()
                .withConnectTimeout(Duration.ofMillis(500))
                .withReadTimeout(Duration.ofSeconds(2));
        return builder
                .baseUrl("http://inventory")
                .requestFactory(ClientHttpRequestFactoryBuilder.detect().build(settings))
                .build();
    }
}

For Feign through Spring Cloud OpenFeign, the settings are per client, with default applying to all:

src/main/resources/application.yamlYAML
spring:
  cloud:
    openfeign:
      client:
        config:
          default:
            connect-timeout: 1000
            read-timeout: 3000
          inventory:
            read-timeout: 2000

Feign values are in milliseconds and the named block overrides the default for the client called inventory. A frequent trap: in older setups the Ribbon or load balancer timeouts silently override the Feign ones, and the property you edited has no effect. Confirm the value actually in force by provoking it: point the client at an endpoint that sleeps longer than the timeout and check how long the call takes to fail.

Shell
time curl -s -o /dev/null http://localhost:8080/test-slow-dependency
Text
real	0m2.043s

A failure after 2.04 seconds confirms the 2-second read timeout is applied. A call that hangs for 30 seconds or a minute means a default (or a different layer's value) is in force. The reference for the clients is in the Spring documentation on REST clients and the Spring Cloud OpenFeign reference.

Choose values from a budget

A timeout is a promise about how long A will wait, and it has to fit inside what A's own caller will wait. Work from the outside in. Suppose the user-facing request has a 5-second budget, A does 200 ms of its own work and calls B and C one after the other:

StepBudget
User's tolerance (and the gateway's timeout)5,000 ms
A's own work and serialization200 ms
Safety margin300 ms
Remaining for dependencies4,500 ms
B (typical 80 ms, p99 400 ms) read timeout1,000 ms
C (typical 150 ms, p99 900 ms) read timeout2,000 ms

The timeouts add to 3,000 ms, within the 4,500 ms available, so even if both stall, A answers (with an error or a fallback) before its own caller gives up. Pick each read timeout a little above the dependency's p99 under normal load, not its average and not an arbitrary 30 seconds. If B's p99 is 400 ms, a 1,000 ms timeout tolerates a bad moment and cuts off a hang. The sum rule is the part that is skipped: three dependencies with 5-second timeouts called sequentially can take 15 seconds to fail, long after the user has left. If A calls B and C in parallel, the slowest sets the bound, not the sum. Keep the connect timeout small (a few hundred milliseconds to a second), because establishing a connection inside a data center is fast, and a long wait means the host is unreachable.

Use the same reasoning for the layers. The proxy or gateway in front of A has its own timeout (often 30 or 60 seconds by default), and A's clients need to be well inside it, otherwise the gateway answers 504 while A is still waiting. Work out the numbers once, write them next to the configuration, and revisit them when the dependency's latency changes.

Find why B is slow

Raising the timeout hides the symptom. Look at B. Start from the side of the failing request: which calls time out, how often and when? Chart the call durations from A's metrics (Micrometer's http.client.requests timer, tagged by URI and outcome) and the same endpoint on B's server side. The pattern tells you where to look:

PatternSuggests
Periodic bursts at the same minute each hourA scheduled job on B, a cache refresh or a backup
Only after a deployCold JVM: classes loading, JIT warming; see below
Grows with loadB is saturated: thread pool, database connections, CPU
One instance of B onlyA bad node, noisy neighbor or an instance in a GC spiral
Always exactly the timeout valueB is not responding at all: a deadlock, a lost connection, a dropped packet

Then use B's own diagnostics. Take thread dumps of B during a slow period and read what the request threads are doing, per using jcmd and jstack safely in production. A pool at its limit throws the rejection described in RejectedExecutionException from a full thread pool; a database pool starved for connections is HikariPool connection is not available. Check B's garbage collection log for long pauses that line up with the timeouts, using reading GC logs to find the real problem. A freshly started instance is slow on its first requests because the JVM has not warmed up; the mechanism is in how the JVM runs bytecode, and the cure is warm-up before it receives traffic and a readiness probe that waits for it.

Add the timeout and its context to A's error so the next incident is shorter. Log which dependency, which URL template, how long it waited and the trace id. A bare Read timed out with no target costs an hour.

Retries and breakers, carefully

Retrying a timed-out call looks free and is not. A retry doubles the load on a dependency that is already struggling, and when many callers retry together the dependency never recovers (a retry storm). Retry only when it is safe and useful:

  • Only idempotent calls. A GET is safe to repeat. A POST that creates something may already have succeeded on B before A timed out; a retry then creates a duplicate unless the call carries an idempotency key, as covered in making REST endpoints idempotent.
  • Few attempts, with backoff and jitter. Two retries at most, with exponentially growing, randomized delays, so callers do not synchronize.
  • Inside the budget. The sum of the attempts and delays must still fit within the request's budget; three attempts at a 2-second timeout is 6 seconds.
  • Retry the right errors: connect failures and 503, not 400.

Spring Framework 7 includes a retry facility:

java
var policy = RetryPolicy.builder()
        .maxRetries(2)
        .delay(Duration.ofMillis(200))
        .multiplier(2.0)
        .jitter(Duration.ofMillis(100))
        .build();

Stock stock = new RetryTemplate(policy).execute(() -> inventory.get().uri("/stock/{sku}", sku)
        .retrieve().body(Stock.class));

A circuit breaker (Resilience4j, or Spring Cloud Circuit Breaker) complements retries: after a run of failures it opens and fails calls immediately, for a cool-off period, giving B room to recover and keeping A's threads free. Pair it with a fallback that makes sense, a cached value or a degraded response, and never with a fallback that hides the failure. Concurrency limits (a bulkhead) stop a slow dependency from taking all of A's threads: a semaphore of 20 concurrent calls to B means the 21st fails fast. Without any of these, a read timeout of 3 seconds with 200 request threads and a dead B holds the entire pool within a second of load.

Where this does not apply

If the exception is ConnectException or Connection refused, the problem is reachability, not slowness, and the checks are different. If A's timeout fires at exactly a round value like 30 seconds, a default or an infrastructure timeout (load balancer, service mesh sidecar) is probably responsible, and A's configuration is not what you need to change. A service mesh with its own retry and timeout policy multiplies with the application's: three layers of three retries make nine attempts, so decide which layer owns resilience. And for long-running operations, a synchronous call with a short timeout is the wrong shape; accept the request and answer later, as in the long-running operations pattern.

Share

Keep reading

Get new posts by email

We send one email when a new post is published. No spam.

Unsubscribe in one click. No spam, ever.