WebSockets Behind a Load Balancer

Analogy categoryAnalogies

The HTTP Upgrade Handshake Through a Proxy

A WebSocket connection begins life as a plain HTTP request carrying Connection: Upgrade and Upgrade: websocket headers, along with a Sec-WebSocket-Key. If the server agrees, it responds with status 101 Switching Protocols and the same TCP connection is repurposed for the full-duplex WebSocket framing from then on. Any load balancer or reverse proxy sitting in between has to actively understand and forward this handshake correctly; a proxy that strips the Upgrade header, buffers the response, or doesn't recognize 101 as a valid terminal status will silently break every WebSocket connection while leaving normal HTTP traffic completely unaffected, which is why WebSocket outages often go unnoticed until someone specifically tests real-time features.

🏏

Cricket analogy: It's like the third umpire needing a specific hand signal from the on-field umpire to switch from live coverage to a DRS review; if the broadcast relay doesn't recognize that signal, the review protocol never activates even though the regular feed looks fine.

Layer 4 vs Layer 7 Load Balancing

Load balancers operate at different layers, and the choice matters a lot for WebSockets. A Layer 4 (TCP) load balancer forwards raw bytes without understanding HTTP at all; it's simple and fast, and it naturally handles the Upgrade handshake because it never inspects it, but you lose L7 features like path-based routing, header inspection, and easy TLS termination at the edge. A Layer 7 (HTTP-aware) load balancer, such as AWS's Application Load Balancer or Nginx configured as a reverse proxy, understands and terminates HTTP, which lets it route /ws paths to specific backend pools and terminate TLS centrally, but it must be explicitly configured to recognize and pass through the Upgrade/101 handshake rather than treating the connection as a normal short-lived HTTP request. Getting this wrong is the single most common cause of 'WebSockets work locally but fail in production'.

🏏

Cricket analogy: It's like the difference between a stadium's basic PA system that just relays raw audio everywhere (Layer 4) versus a smart broadcast van that reads the commentary script and routes Hindi commentary to one channel and English to another (Layer 7); the smart system needs explicit configuration to also pass through DRS review audio correctly.

Timeouts and Idle Connections

Most load balancers enforce an idle timeout that closes a connection if no bytes flow for a configured period, and this timeout is almost never tuned by default for WebSockets, which can sit silent for long stretches between application messages. AWS's Application Load Balancer, for example, defaults to a 60-second idle timeout; if your chat app only sends messages when a user types, a quiet connection gets forcibly closed well before the user notices anything is wrong, and the client has to detect the drop and reconnect. The standard fix is twofold: raise the load balancer's idle timeout to a comfortable margin (many teams use 300+ seconds), and implement application-level ping/pong heartbeats sent every 20-30 seconds so the connection never actually goes idle from the proxy's point of view, which also lets you detect truly dead connections faster than TCP's own timeouts would.

🏏

Cricket analogy: It's like a stadium's Wi-Fi router that disconnects any device idle for 60 seconds; a spectator checking the scoreboard app only during boundaries gets silently kicked off between overs unless the app pings the router periodically to stay alive.

nginx

# Nginx configured as a WebSocket-aware reverse proxy
map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

server {
    listen 443 ssl;
    server_name ws.example.com;

    location /ws/ {
        proxy_pass http://websocket_backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_set_header Host $host;

        # Raise idle timeout well above your heartbeat interval
        proxy_read_timeout 300s;
        proxy_send_timeout 300s;
    }
}

Don't rely on default timeouts and assume they'll 'just work'. AWS ALB idle timeout defaults to 60 seconds and Google Cloud's HTTP(S) Load Balancer historically defaulted to similarly short windows; both must be explicitly raised for WebSocket target groups/backends, and even then you should implement application-level heartbeats since some intermediate NATs and corporate proxies enforce their own shorter timeouts you can't configure.

Sticky Sessions at the LB

Because the Upgrade request and the resulting long-lived connection must reach the same backend instance, most load balancers offer session affinity (sticky sessions), typically implemented by setting a cookie on first contact and consistently routing subsequent requests with that cookie to the same target. For WebSockets, though, the important routing decision happens exactly once, at the initial handshake, so the affinity mechanism only needs to correctly pin that one request; some load balancers use source-IP hashing instead of cookies, which is simpler but can cluster traffic unevenly if many clients share a NAT gateway (common on corporate networks and some mobile carriers). Whichever mechanism you choose, it should be paired with health checks that detect a backend going unhealthy and stop routing new handshakes there, even though existing sockets on that backend will still need to be drained separately.

🏏

Cricket analogy: It's like a stadium issuing a wristband at the gate that determines which specific entry gate you use for the rest of the match; if thousands of fans arrive on one shared shuttle bus (like a shared NAT), they might all get funneled through the same gate, causing uneven crowding.

WebSockets start as an HTTP request with Upgrade/Connection headers and a 101 Switching Protocols response.
Load balancers must explicitly support and forward this handshake or connections silently fail.
Layer 4 (TCP) load balancers pass bytes through blindly and naturally handle the handshake but lack HTTP-aware routing.
Layer 7 (HTTP-aware) load balancers need explicit Upgrade-header configuration to avoid breaking WebSocket traffic.
Default idle timeouts (e.g., 60s on AWS ALB) are usually too short and must be raised for WebSocket backends.
Application-level ping/pong heartbeats keep connections active and detect dead sockets faster than relying on TCP timeouts alone.
Sticky sessions pin the one-time handshake to a backend; IP-hash affinity can cluster unevenly behind shared NATs.