Savepoints

Analogy categoryAnalogies

Savepoints vs. Checkpoints

A savepoint is structurally similar to a checkpoint — both use the same barrier-based snapshot mechanism — but savepoints are triggered manually (or via the REST/CLI API) rather than automatically on a schedule, are stored in a self-contained, canonical binary format independent of the configured state backend, and are meant to be retained indefinitely rather than cleaned up automatically. Where checkpoints exist purely for automatic failure recovery, savepoints exist for deliberate, operator-initiated actions: upgrading application code, changing parallelism, migrating to a different cluster, or forking a job to test a new code path against production state.

🏏

Cricket analogy: A checkpoint is like the automatic scoreboard update after every over, while a savepoint is like a curator deliberately archiving the full scorecard and pitch report after a historic Test match for the museum — intentional, curated, and meant to last.

Triggering and Restoring a Savepoint

Savepoints are triggered with the flink savepoint CLI command (or the equivalent REST API call), specifying the running job ID and a target directory; Flink drains the barrier through the pipeline exactly as it would for a checkpoint, then writes the canonical snapshot to that directory and returns its path once complete. Restoring is equally explicit: flink run -s <savepoint-path> starts a new job execution from that savepoint, and Flink matches operator state back to the new job graph using stable operator UIDs — which is why assigning explicit .uid() values to every stateful operator is considered essential production practice, since Flink's auto-generated IDs shift whenever the job graph topology changes.

🏏

Cricket analogy: A team requesting an official pitch report from the curator before a crucial final is a deliberate request-and-response action, just as triggering a savepoint requires explicitly asking for a snapshot and specifying where it should be filed.

bash

# Trigger a savepoint for a running job
flink savepoint <jobId> s3://flink-savepoints/my-job/

# Stop the job gracefully with a final savepoint
flink stop --savepointPath s3://flink-savepoints/my-job/ <jobId>

# Restore a new job execution from that savepoint
flink run -s s3://flink-savepoints/my-job/savepoint-abc123 my-job.jar

Using Savepoints for Upgrades and Rescaling

Because savepoints decouple state from both the runtime state backend and the exact parallelism at capture time, they're the standard mechanism for zero-data-loss application upgrades: stop the job with a savepoint, deploy the new JAR (which may add new operators, remove obsolete ones, or change logic on existing stateful operators), and restore from the savepoint — Flink matches state by UID, silently drops state for UIDs no longer present, and lets new stateful operators start from their configured default state. The same mechanism supports rescaling: restoring a savepoint with a different -p parallelism value causes keyed state's key groups to redistribute across the new number of subtasks automatically.

🏏

Cricket analogy: A squad rotation between series keeps continuity by carrying over each returning player's stats while new call-ups start with a fresh record and dropped players' stats are archived, mirroring how a savepoint restore matches state by UID.

Use flink stop (not flink cancel) when you want a guaranteed final, complete savepoint before shutdown — stop performs a synchronous savepoint and only terminates the job once it succeeds, whereas cancel followed by a separate savepoint call has a race condition risk.

State Compatibility Across Restores

State compatibility during a savepoint restore is governed by the serializer used for each state descriptor: Flink's built-in POJO and Avro serializers support certain schema evolution rules — adding or removing fields on a POJO, for example — but changing a field's type or renaming a state variable's name (which is used as its identifier) breaks compatibility and causes the restore to fail unless state migration or a custom TypeSerializerSnapshot is provided. Testing savepoint restores against a staging environment before every production code change involving state schema is standard practice for exactly this reason.

🏏

Cricket analogy: Adding a new optional column to a scorecard template (like 'strike rate') is a safe evolution that old scorecards can still be read against, but renaming the 'runs' column to 'score' breaks every historical scorecard's compatibility, like renaming a state variable.

Never rely on Flink's auto-generated operator UIDs for anything beyond quick local testing — assign explicit .uid("my-stable-name") calls to every stateful operator, because reordering, adding, or removing operators in your topology reshuffles the auto-generated IDs and can silently orphan state on the next savepoint restore.

Savepoints use the same barrier-based mechanism as checkpoints but are manually triggered and stored in a canonical, backend-independent format.
Savepoints are meant for deliberate operations: upgrades, rescaling, cluster migration, and forking a job's state for testing.
flink stop performs a synchronous savepoint and only terminates the job after it succeeds, unlike flink cancel.
Restoring matches state to operators via stable UIDs — always assign explicit .uid() values to stateful operators in production.
Rescaling parallelism during a savepoint restore automatically redistributes keyed state's key groups.
State schema evolution rules depend on the serializer; some changes (adding fields) are safe, others (renaming, retyping) break compatibility.
Test savepoint restores in staging before deploying state-schema-affecting changes to production.