What are Retries and Exponential Backoff, and why use jitter?
Learn how exponential backoff and jitter prevent retry storms, why immediate retries worsen outages, and how to retry safely and idempotently.
Expected Interview Answer
Retries automatically re-attempt a failed request on the assumption the failure is transient, and exponential backoff spaces those retries out with exponentially increasing delays (e.g. 1s, 2s, 4s, 8s) so repeated failures do not hammer an already-struggling dependency; jitter adds randomness to those delays to prevent many clients from retrying in synchronized bursts.
A naive retry loop that fires immediately after every failure can turn a brief blip into a self-inflicted denial-of-service, because every failed client retries at once and doubles or triples the load exactly when the dependency is weakest. Exponential backoff fixes the timing by growing the wait between attempts geometrically, giving the failing service room to recover instead of being retried into the ground. But if every client computes the exact same backoff schedule from the same failure timestamp, they still retry in lockstep waves — this is the thundering herd problem. Jitter breaks that synchronization by adding a random offset to each computed delay (full jitter picks a random value between 0 and the computed backoff, decorrelated jitter factors in the previous delay), spreading retries out smoothly over time. Retries should also be capped with a maximum attempt count or maximum total elapsed time, and only applied to idempotent operations or ones made safe via idempotency keys, since retrying a non-idempotent write can duplicate side effects.
- Recovers automatically from transient network blips without manual intervention
- Exponential growth prevents retries from overwhelming a recovering service
- Jitter avoids synchronized retry storms across many concurrent clients
- Capped attempts and idempotency checks prevent duplicate side effects and runaway retries
AI Mentor Explanation
Retries with backoff are like a bowler who, after a no-ball is called, does not immediately sprint back in and bowl again at the same pace — the umpire has them pause a beat, and if it happens again, pause longer each time before the next attempt. Jitter is like each bowler in the attack choosing a slightly different pause length instead of every bowler in the team resetting on the exact same countdown, so the umpire is not swamped reviewing five identical deliveries at once. Waiting longer after repeated failures gives the situation room to settle rather than compounding the problem. A capped number of retries mirrors an umpire only allowing so many reviews before the decision stands.
Step-by-Step Explanation
Step 1
Detect a transient failure
Distinguish retryable errors (timeouts, 503s, connection resets) from non-retryable ones (4xx validation errors).
Step 2
Compute exponential backoff
Grow the wait geometrically, e.g. base * 2^attempt, up to a maximum delay cap.
Step 3
Apply jitter
Randomize the computed delay (full or decorrelated jitter) so concurrent clients do not retry in lockstep.
Step 4
Cap attempts and check idempotency
Stop after a max attempt count or elapsed time, and only retry operations that are safe to repeat.
What Interviewer Expects
- Understanding of why naive immediate retries can worsen an outage
- Ability to explain exponential growth of the retry delay
- Clear explanation of jitter and the thundering herd problem it solves
- Awareness that retries must be bounded and limited to idempotent operations
Common Mistakes
- Retrying immediately with no delay, amplifying load on a struggling service
- Using exponential backoff without jitter, still causing synchronized retry waves
- Retrying non-idempotent operations and creating duplicate side effects
- Forgetting to cap the maximum number of attempts or total retry duration
Best Answer (HR Friendly)
“When a request fails, instead of giving up immediately or hammering the service with instant retries, we wait a bit before trying again, and we double that wait time after each subsequent failure — that is exponential backoff. We also add a little randomness to that wait, called jitter, so that if thousands of clients failed at the same time, they don’t all retry in the exact same instant and overwhelm the service again the moment it recovers.”
Code Example
attempt=0
max_attempts=5
base_delay=1
max_delay=30
until curl -sf https://api.example.com/orders -X POST -d "$PAYLOAD"; do
attempt=$((attempt + 1))
if [ "$attempt" -ge "$max_attempts" ]; then
echo "Giving up after $attempt attempts" >&2
exit 1
fi
capped=$(( base_delay * (2 ** attempt) ))
[ "$capped" -gt "$max_delay" ] && capped=$max_delay
jittered=$(( RANDOM % capped + 1 )) # full jitter: random(0, capped)
echo "Retry $attempt in ${jittered}s"
sleep "$jittered"
doneFollow-up Questions
- What is the thundering herd problem and how does jitter solve it?
- Why should you only retry idempotent operations?
- What is the difference between full jitter and decorrelated jitter?
- How would you decide the maximum number of retry attempts for a payment API call?
MCQ Practice
1. What does exponential backoff do between retry attempts?
Exponential backoff increases the wait time geometrically (e.g. doubling) after each failed attempt, giving the failing service time to recover.
2. What problem does adding jitter to backoff delays solve?
Jitter randomizes each client's delay so concurrent retries spread out over time instead of arriving in synchronized waves.
3. Why should retries generally be limited to idempotent operations?
Retrying a non-idempotent write (like charging a card) without deduplication can cause the same side effect to happen multiple times.
Flash Cards
What is exponential backoff? — Increasing the delay between retry attempts geometrically after each failure.
What is jitter and why use it? — Randomizing the backoff delay to prevent synchronized retry storms (thundering herd) across clients.
Should you retry all failed requests? — No — only retryable transient errors on idempotent operations, with a capped attempt count.
What is the thundering herd problem? — Many clients retrying at the exact same moment, overwhelming a service right as it recovers.