Execution-history retention

Every task in a durable workflow is a persisted event row. Without retention, executions and execution_events grow without bound — the table sizes become a function of everything your deployment has ever run. Flux ships a server-side retention job (flux/retention.py) that deletes old terminal executions on a schedule.

It is off by default, so upgrades never silently remove history. Enable it in production:

[flux.retention]
enabled = true
retention_days = 30    # age (days since last event) before a terminal execution is deleted
sweep_interval = 3600  # seconds between sweeps
batch_size = 500       # executions deleted per transaction

Env vars: FLUX_RETENTION__ENABLED, FLUX_RETENTION__RETENTION_DAYS, FLUX_RETENTION__SWEEP_INTERVAL, FLUX_RETENTION__BATCH_SIZE.

What gets deleted

A sweep deletes executions that are:

• Terminal — state COMPLETED, FAILED, or CANCELLED. Running, paused, and scheduled executions are never touched, no matter how old.
• Older than retention_days — measured from the execution’s most recent event, not its creation time. A long-running execution that finished yesterday is safe even if it started months ago.

Dependent rows go with the execution: its execution events, agent sessions, and approval requests are deleted explicitly in the same pass, so no orphans accumulate on backends that don’t enforce cascades.

How the sweep runs

• The job runs inside the server process, sweeping every sweep_interval seconds (default hourly).
• Each sweep deletes in batches of batch_size, one transaction per batch. A crash mid-sweep loses nothing; the next sweep resumes where the last one stopped. Batching also keeps lock hold times and WAL bursts small on busy databases.
• Multi-replica deployments need no coordination from you: each sweep cycle takes a PostgreSQL advisory lock, so exactly one replica sweeps at a time; the others skip the cycle and log at debug. A dead holder’s lock auto-releases. (On SQLite the lock is a no-op — SQLite is single-node anyway.)
• Successful sweeps log at INFO: Retention sweep deleted N execution(s).

Choosing retention_days

The trade is disk versus audit history. Questions to answer before shortening it:

• How far back do incident investigations actually reach? flux execution show on a purged execution returns nothing.
• Do compliance requirements mandate a minimum? If so, consider exporting events to your warehouse before they age out — retention deletes, it does not archive.
• Are transient workflows carrying your high-volume traffic? They persist only ~4 event rows per execution, which relieves growth pressure at the source and lets you keep durable history longer.

Backups interact the obvious way: a restore resurrects rows the retention job had deleted, and the next sweep deletes them again. See Backups and restore.

What’s next

• Backups and restore — snapshot the event log before you shorten retention.
• Capacity planning — the storage-growth math retention bounds.
• Durable vs transient workflows — reducing event volume at the source.
• Server and worker settings — the [flux.retention] reference entries.