A Call Without a Deadline Holds the Process

A missing timeout is not a slow request. It is a request that has not decided to stop. The socket sits in the worker until the kernel gives up, which on a default TCP stack is many minutes. During that time the worker is busy. It cannot take another request. A handful of calls to a dead dependency is enough to fill the pool, and the site looks down while the process table says everything is running.

The failure you wanted was a fast error. The failure you get is a queue.

Put the deadline on the call

The timeout belongs next to the call, in the same place you would put an error check. A process-wide default is a backstop. It is not the deadline for this call, because this call knows how long it can wait and the process does not.

In Go the deadline is a context. The client does not invent one if you forget. http.Get uses http.DefaultClient, whose timeout is zero, which means it waits until the connection dies on its own.

ctx, cancel := context.WithTimeout(ctx, 2*time.Second)
defer cancel()

req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
    return Result{}, err
}

resp, err := client.Do(req)
if err != nil {
    return Result{}, fmt.Errorf("get %s: %w", url, err)
}
defer resp.Body.Close()

Two seconds is not a magic number. It is the budget for this dependency. A search that must answer inside a page render gets a short one. A report the user is willing to wait for gets a longer one. The number is a product decision written down where the call is made.

Canceling the context does not always cancel the work on the other side. It stops this side from waiting. That is the part that protects the worker. If the other side also needs to stop, it has to watch the same deadline, which is why you pass ctx into your own functions instead of starting a fresh context.Background() at the boundary.

The same hole in PHP

Laravel's HTTP client waits thirty seconds if you say nothing. Thirty seconds times a pool of workers is a queue of people looking at a spinner. Set the timeout where you make the call, and decide what a timeout means before you write the retry.

$response = Http::timeout(2)
    ->connectTimeout(1)
    ->get($url);

if ($response->failed()) {
    report(new UpstreamException($url, $response->status()));

    return Cache::get($cacheKey, $fallback);
}

timeout is the whole transfer. connectTimeout is the handshake. A service that accepts the TCP connection and then says nothing trips the first and not the second. You want both. A connect timeout of one second fails fast when the host is unreachable. A transfer timeout of two seconds fails when it accepted the socket and stalled.

The fallback matters as much as the deadline. A timeout that becomes a 500 teaches the user nothing except that the page is broken. If the last good value is still true enough, serve it and say so. If it is not, fail the request with a status you chose, not with a worker that is still blocked.

What not to do

Do not retry a timeout as if it were a typo. The upstream is already late. A second attempt spends the budget twice, and if every worker does it you have built a small outage into a larger one. Retry a connection refused, or a 503, with a limit and some jitter. A timeout has already used its turn.

Do not set the deadline shorter than the work. A two-second timeout around a query that legitimately takes three seconds is not strictness. It is a guaranteed error, and the retries that follow make the query take longer. Measure the normal case, then put the deadline past it with room for a bad day. A deadline that fires on a healthy call is the wrong deadline.

Do not log the body you did not finish reading. After a cancel, reads return an error. The useful line is the URL, the deadline, and that the deadline won. A stack trace that starts in the JSON decoder is a distraction from the fact that you stopped on purpose.

A long wait is still a decision

Some calls should be slow. A payment capture, a video transcode, a migration. Those do not belong on the request that is painting a page. Move them to a job that has its own deadline, longer and explicit, and let the page return. The worker that renders HTML should be free in tens of milliseconds, and anything that can hold it longer needs a deadline written next to the call.

I would rather fail a dependency in two seconds than explain, an hour later, why every worker is stuck on the same socket.