Checkpoints & storage

Three kinds of storage, with genuinely different durability. Knowing which is which avoids the only truly unpleasant surprise on this platform.

	Lives in	Survives restart	Survives suspend	Survives environment delete
Flink state (checkpoints, savepoints)	Your S3 bucket	✅	✅	✅
Log history	Your S3 bucket	✅	✅	✅
Kafka messages	Your S3 bucket	✅	❌	❌

Kafka is the odd one out, and the reason is worth a sentence: its messages are written to your bucket like everything else, but the index that makes them readable — the cluster's own metadata — is compute. Suspend rebuilds the cluster empty, so the messages do not come back even though the bytes were never on a disk that died.

The pattern: anything you can point a tool at in object storage is yours and persists; anything that only a running cluster can interpret goes when that cluster goes.

Checkpoints

Flink writes a checkpoint periodically so a crashed job resumes where it stopped rather than from the beginning. They are automatic once enabled, and they are what makes a stateful streaming job trustworthy.

They land in your own S3 bucket. We do not hold your state — the credentials the job uses are scoped to your bucket and issued to the pod itself, so there are no keys in any manifest and none in our database.

You can see what has actually been written from the job page. It is a read-only listing: the objects are Flink's internal format and mean nothing outside it, and deleting a checkpoint from under a running job is a way to lose data with no undo.

Savepoints

A savepoint is a checkpoint you asked for, at a moment you chose. Use one before upgrading a job: Savepoint takes it, and Upgrade restarts the new version from it, so the job keeps its state across a code change.

Upgrading always goes through a savepoint. The alternative — a stateless upgrade — brings the job back healthy and empty, having quietly forgotten everything it knew. That is the most expensive mistake available in a streaming platform, so we do not offer it as a default.

Log history

Shipped continuously to your bucket and searchable from the job's Logs → History tab.

Retention is yours to set, from a day to two years. Longer retention costs you object storage, which is cheap — it does not cost broker disk or compute, which is why we can offer two years at all.

Kafka messages

Your S3 bucket, with the retention set per topic (up to 7 days). The brokers themselves hold no data: they are stateless, and every byte is in the bucket before a produce is acknowledged. That is what makes losing a broker a non-event — and what makes writes take hundreds of milliseconds rather than single digits. See Connect your app to Kafka.

Suspending an environment deletes Kafka data

Suspend removes the compute, and the cluster's own metadata goes with it — resume rebuilds the cluster from your declarative configuration rather than restoring it. Your topics, credentials and configuration come back. The messages do not, even though they were in object storage rather than on a broker disk: a rebuilt cluster has no index into them.

This is deliberate. Suspend exists to take an environment's cost to nearly zero. If messages need to outlive a suspend, write them from Flink to S3 in a format you own.

Deleting an environment

Terminating removes the compute and the cluster. Your S3 bucket is yours and is not deleted with it — checkpoints, savepoints and log history remain, and you can retrieve them with your own AWS credentials.

That is intentional: the one thing you cannot recreate is your data.