How Do You Rebalance Shards Without Downtime?

Learn how to rebalance database shards live using dual-write, backfill, and verified cutover with zero downtime.

hardQ141 of 224 in System Design Est. time: 6 minsLast updated: July 2026

Expected Interview Answer

Rebalancing shards without downtime means gradually migrating a subset of data to new or less-loaded shards while keeping both the old and new locations correctly serving reads and writes, typically using a dual-write or replicate-then-cutover approach with a routing layer that flips ownership only after the migrated data is verified consistent.

The core technique is to pick the keys or ranges that need to move, start replicating their writes to the destination shard while the source shard keeps serving as the source of truth, and backfill historical data in the background. Once the destination is caught up and verified, the routing layer atomically flips which shard owns those keys, and only after a safety window with no more reads hitting the old location is the source data deleted. Using consistent hashing with virtual nodes minimizes how much data needs to move for any single rebalance, and having an abstraction layer between application code and physical shard placement (a routing table or proxy) means clients never need to know a migration happened. Throttling the migration’s throughput protects live traffic from being starved by the backfill process.

Keeps the system serving reads and writes throughout the entire migration
Consistent hashing minimizes the fraction of data that must move per rebalance
Dual-write plus verified cutover avoids losing or duplicating data mid-migration
A routing abstraction keeps rebalancing invisible to application clients

AI Mentor Explanation

Rebalancing shards is like moving part of an overcrowded stand’s ticket holders to a newly opened stand mid-match without ever stopping play. Stewards start directing new arrivals with matching tickets to the new stand while the old stand keeps serving everyone already seated there, and only once the new stand is fully staffed and verified does the gate officially switch which stand a ticket range belongs to. Nobody loses their seat and nobody is double-counted because the switch happens atomically after verification. That live, verified hand-off is exactly how a database rebalances shards without downtime.

Step-by-Step Explanation

Step 1
Select the migration set
Identify which key ranges or virtual nodes are moving, chosen to minimize total data movement (e.g. via consistent hashing).
Step 2
Start dual-write and backfill
New writes for the migrating keys are mirrored to the destination shard while historical data is copied in the background.
Step 3
Verify consistency
Compare source and destination checksums or row counts to confirm the destination is fully caught up before cutover.
Step 4
Atomically flip routing and clean up
Update the routing table to point at the new shard, then after a safety window delete the now-redundant data from the source.

What Interviewer Expects

Describes a replicate-then-verify-then-cutover pattern, not a stop-the-world copy
Mentions a routing layer or metadata service that abstracts shard placement from clients
Recognizes consistent hashing reduces how much data must move per rebalance
Discusses throttling migration throughput to protect live production traffic

Common Mistakes

Proposing to simply stop writes during the entire migration window
Forgetting to verify data consistency before flipping ownership
Not considering how clients discover the new shard location (routing/metadata layer)
Ignoring the need to throttle backfill so it does not starve live traffic

Best Answer (HR Friendly)

“Rebalancing shards without downtime means moving a slice of data to a new location while the system keeps running normally. You start copying the data in the background and mirroring new writes to both the old and new spot, then once you’ve confirmed the new spot is fully caught up and correct, you flip a switch so all future requests go there instead. Because that switch only happens after verification, users never notice the move happening.”

Code Example

Dual-write migration with verified cutover (pseudo-code)

def migrate_shard_range(key_range, source, dest, router):
    backfill(source, dest, key_range)          # copy historical rows
    router.enable_dual_write(key_range, dest)  # mirror new writes to dest too

    while not is_caught_up(source, dest, key_range):
        sleep(POLL_INTERVAL)

    if not verify_consistency(source, dest, key_range):
        raise MigrationError("checksum mismatch, aborting cutover")

    router.flip_ownership(key_range, dest)      # atomic: reads now go to dest
    wait(SAFETY_WINDOW_SECONDS)
    router.disable_dual_write(key_range)
    delete_range(source, key_range)             # reclaim space on old shard

Follow-up Questions

How would you handle a write that arrives mid-migration for a key that has already been cut over?
What happens if the destination shard falls behind during backfill on a high-write-volume table?
How does consistent hashing with virtual nodes reduce the blast radius of a rebalance?
How would you roll back a rebalance if verification fails after cutover?

MCQ Practice

1. What is the safest general pattern for rebalancing shards in a live system?

2. Why is a routing/metadata layer important during shard rebalancing?

3. Why throttle the backfill process during a shard migration?

Flash Cards

1 / 4

How Do You Rebalance Shards Without Downtime?

Expected Interview Answer

AI Mentor Explanation

Step-by-Step Explanation

Select the migration set

Start dual-write and backfill

Verify consistency

Atomically flip routing and clean up