Logging in Nginx

Analogy categoryAnalogies

Access Logs and the log_format Directive

Nginx writes one access log line per completed request, controlled by the access_log directive and formatted according to a named log_format. The built-in combined format captures remote address, timestamp, request line, status code, bytes sent, referer, and user agent, but production deployments almost always define a custom format that adds fields like $request_time (total time to serve the request) and $upstream_response_time (time the backend took), which are essential for diagnosing whether slowness originates in Nginx or in the application server behind it.

🏏

Cricket analogy: It's like a match scorecard's default columns (runs, wickets, overs) being the 'combined' format, while a serious analytics team adds custom columns like strike rate under pressure and bowling economy in the death overs — the extra fields that actually diagnose performance.

Distinguishing request_time from upstream_response_time

$request_time measures the total elapsed time from when Nginx received the first byte of the request to when it finished sending the last byte of the response, including any time spent reading a slow client's request body. $upstream_response_time measures only the time the proxied backend took to respond, and when a request goes through multiple upstream attempts (due to retries), it's logged as a comma-separated list of each attempt's duration. A large gap between the two — high request_time but low upstream_response_time — usually points to network latency or a slow client, not application code.

🏏

Cricket analogy: It's like the difference between the total time a bowler takes to complete an entire over including a mid-over drinks break ($request_time) versus just the time the ball is actually in the air per delivery ($upstream_response_time); a big gap points to the break, not the bowling.

Error Logs and Log Levels

The error_log directive controls Nginx's own diagnostic logging, separate entirely from access logs, and takes a severity level such as debug, info, notice, warn, error, crit, or emerg — each level includes all messages at that severity and above. Setting error_log /var/log/nginx/error.log warn; in production is typical, since debug level requires a special debug build of Nginx and produces enormous volumes of output suitable only for short-lived troubleshooting sessions, never left running long-term.

🏏

Cricket analogy: It's like a coach's notes ranging from casual observations up to a formal disciplinary report; you don't file a formal report for every minor fielding lapse, only escalate what's actually severity-worthy, just as error_log filters by level.

nginx

http {
    log_format perf '$remote_addr - $host [$time_local] "$request" '
                     'status=$status bytes=$body_bytes_sent '
                     'req_time=$request_time upstream_time=$upstream_response_time '
                     'ref="$http_referer" ua="$http_user_agent"';

    server {
        listen 80;
        server_name example.com;

        access_log /var/log/nginx/example.access.log perf buffer=32k flush=5s;
        error_log  /var/log/nginx/example.error.log warn;

        location /api/ {
            proxy_pass http://backend_upstream;
            # Skip logging noisy health checks to keep the access log useful
            location = /api/health {
                access_log off;
                proxy_pass http://backend_upstream;
            }
        }
    }
}

Use buffer=32k flush=5s on access_log to batch writes and reduce disk I/O on high-traffic servers, and pair it with access_log off; on noisy, low-value endpoints like health checks so real traffic patterns aren't drowned out in log analysis.

Leaving error_log at debug level in production is a serious operational risk: it can generate gigabytes of output per hour, fill the disk, and even measurably slow down request handling since every log line requires a synchronous write unless explicitly buffered. Reserve debug level for short, targeted troubleshooting windows only.

access_log records one line per completed request; error_log records Nginx's own diagnostic messages.
log_format defines custom access log fields; the built-in combined format is a reasonable starting point only.
$request_time is total time serving the request; $upstream_response_time is only the backend's response time.
A large gap between request_time and upstream_response_time usually indicates client or network latency, not app slowness.
error_log severity levels (debug, info, notice, warn, error, crit, emerg) are cumulative — each includes all levels above it.
debug-level logging requires a special build and floods output; it should never run long-term in production.
access_log off; and buffer/flush parameters help control log volume and I/O on high-traffic endpoints.