Connection and Buffer Tuning

Analogy categoryAnalogies

Why Connection and Buffer Tuning Matters

Nginx's default configuration is tuned conservatively for general-purpose use, which means high-traffic deployments often hit invisible ceilings long before the hardware itself is actually saturated. Connection tuning controls how many simultaneous clients Nginx can hold open, while buffer tuning controls how much of each response Nginx keeps in memory versus spilling to slower disk, and getting either wrong produces symptoms like refused connections, 502/504 errors, or unexpectedly high latency under load that looks like a mystery until you inspect the actual settings.

🏏

Cricket analogy: Tuning worker_connections is like a stadium determining how many turnstiles to open for a high-demand IPL final — too few and fans queue outside (connections dropped), too many unstaffed ones and gates sit idle wasting resources.

Worker and Connection Limits

worker_processes auto; spawns one worker process per detected CPU core so Nginx can use all available cores in parallel, while worker_connections in the events block caps how many simultaneous connections each individual worker can handle. Because every connection typically consumes at least one file descriptor, worker_rlimit_nofile must be raised alongside worker_connections or the operating system's own limit becomes the real bottleneck long before Nginx's configured ceiling is reached; multi_accept on; further lets a worker drain its entire accept queue in one pass instead of one connection at a time.

🏏

Cricket analogy: worker_processes auto; is like a tournament automatically assigning one match official per available ground rather than a fixed number regardless of how many grounds are hosting games that day — Nginx spawns one worker per detected CPU core.

nginx

worker_processes auto;
worker_rlimit_nofile 65535;

events {
    worker_connections 4096;
    multi_accept on;
}

http {
    upstream backend {
        server 10.0.0.11:8080;
        server 10.0.0.12:8080;
        keepalive 64;
    }

    server {
        location / {
            proxy_pass http://backend;
            proxy_http_version 1.1;
            proxy_set_header Connection "";

            proxy_buffer_size 8k;
            proxy_buffers 8 16k;
            proxy_busy_buffers_size 32k;
        }
    }
}

Buffers for Proxying

proxy_buffer_size holds the first part of an upstream response, typically just the headers, while proxy_buffers (specified as a count and a size) buffers the remaining response body in memory; proxy_busy_buffers_size then limits how much of that buffered data can be actively in flight to the client at once. When the configured buffers are too small for the actual response size, Nginx falls back to writing the overflow to a temporary file on disk, which is dramatically slower than staying fully in memory and is a common, easy-to-miss source of latency spikes.

🏏

Cricket analogy: proxy_buffer_size handling the first part of the upstream response, typically headers, is like a scorer's small notepad for jotting the immediate over summary, while proxy_buffers handles the larger scorebook holding the full innings detail.

Monitor $upstream_response_time versus $request_time in your log format to see whether latency originates upstream or in Nginx's own buffering/proxying layer before tuning buffer sizes blindly.

Keepalive and Upstream Connections

The keepalive directive inside an upstream block tells Nginx to maintain a pool of already-open TCP connections to backend servers, avoiding the overhead of a fresh handshake for every single proxied request. This only works if the location block also sets proxy_http_version 1.1; and clears the Connection header with proxy_set_header Connection "";, since without both, Nginx falls back to HTTP/1.0-style behavior that closes the upstream connection after every response regardless of what the keepalive directive says.

🏏

Cricket analogy: The keepalive directive in an upstream block keeping connections to backend servers open is like a bowler who stays warmed up at the top of their run-up between overs instead of doing a fresh warm-up sprint before every single delivery.

Setting keepalive in an upstream block requires proxy_http_version 1.1; and proxy_set_header Connection ""; in the location block — omitting either silently disables connection reuse even though the directive appears configured correctly.

worker_processes auto; scales Nginx's worker count to match available CPU cores automatically.
worker_connections and worker_rlimit_nofile must be raised together — increasing one without the other doesn't lift the real ceiling.
proxy_buffer_size handles the response headers; proxy_buffers handles the body, and both should be sized to the actual upstream response profile.
Undersized proxy_buffers force Nginx to buffer to disk, which is significantly slower than staying in memory.
Upstream keepalive connections require proxy_http_version 1.1 and an empty Connection header to actually take effect.
multi_accept on; lets a worker accept all queued connections in one pass instead of one at a time.
Always measure before and after tuning — buffer and connection settings that help one workload can hurt another.