Replays and re-execution

Every Flux execution leaves behind an event log. Each task completion, pause, retry, and failure is recorded before the workflow advances. When you re-run an execution by its ID, Flux reads that log and short-circuits every task that already completed; the task body never runs again, and the recorded output is returned directly. This is replay.

Replay is how Flux resumes paused workflows, recovers from worker crashes, and supports debug re-runs. In each case the mechanism is the same: scan the event log, skip completed tasks, run what remains.

What replay means in practice

Run a workflow:

from flux import ExecutionContext
from flux.task import task
from flux.workflow import workflow


@task
async def fetch_records(source: str) -> list[str]:
    print(f"fetching from {source}")   # expensive, real network call
    return ["rec-1", "rec-2", "rec-3"]


@task
async def process(records: list[str]) -> dict:
    return {"count": len(records), "processed": True}


@workflow
async def etl_pipeline(ctx: ExecutionContext[str]):
    records = await fetch_records(ctx.input)
    return await process(records)


ctx = etl_pipeline.run("warehouse://sales")
print(ctx.execution_id)   # e.g. "c96d7b71..."
print(ctx.output)         # {"count": 3, "processed": True}

Now replay that execution:

ctx2 = etl_pipeline.run(execution_id=ctx.execution_id)
print(ctx2.output)        # {"count": 3, "processed": True}
# "fetching from warehouse://sales" is NOT printed — task body did not run

The two tasks already have TASK_COMPLETED events in the log. Flux finds them, returns the recorded outputs, and the workflow completes immediately without touching the warehouse. The printed side effect never fires.

Inspecting the event log before replaying

Before you replay, inspect the execution to understand what happened:

# Summary (state, input, output)
flux execution show <execution_id>

# Full event log (every TASK_STARTED, TASK_COMPLETED, retries, pauses)
flux execution show --detailed <execution_id>

The detailed output includes one entry per event with its type, source_id (a hash of the task name and arguments), name, value, and timestamp. Reading the log tells you exactly which tasks ran, how many retry attempts fired, and what each task returned or raised.

From the SDK, the same information is on the ExecutionContext.events list:

for event in ctx.events:
    print(event.type, event.name, event.value)

Re-running with a specific workflow version

Flux versions workflows on every flux workflow register. To run an older version, pass --version. This is useful to compare behaviour across releases, roll back a bad deployment, or reproduce a historical execution precisely:

# List all versions
flux workflow versions etl_pipeline

# Re-run the same workflow, pinned to version 2
flux workflow run etl_pipeline '{"source": "warehouse://sales"}' --version 2

From the CLI, --version (short: -v) accepts an integer version number. The server dispatches the execution to a worker that loads the requested version’s source code.

Determinism and what it requires

Replay works because Flux looks up each task invocation by a stable key derived from the task name and its arguments. If the workflow produces the same task calls with the same arguments on every run, Flux finds the corresponding TASK_COMPLETED events and replays them correctly.

This breaks when the workflow function generates arguments non-deterministically. A timestamp or a UUID produced outside a task will differ on replay, so Flux finds no matching event and re-executes the task body:

import uuid

@workflow
async def bad_workflow(ctx: ExecutionContext):
    # This UUID is different every time the function runs.
    # On replay, Flux sees a new task_id and re-runs the task body.
    result = await expensive_task(str(uuid.uuid4()))
    return result

The fix is to generate non-deterministic values inside a task, where the output is captured in the event log:

from flux.tasks import uuid4   # built-in deterministic UUID task

@workflow
async def good_workflow(ctx: ExecutionContext):
    # uuid4() is a @task — its output is recorded and replayed.
    request_id = await uuid4()
    result = await expensive_task(request_id)
    return result

flux.tasks.uuid4 is a built-in task that generates a UUID once and records it. On replay, Flux finds the TASK_COMPLETED event for that task, returns the same UUID without calling uuid.uuid4() again, and passes it to expensive_task, which also replays from its recorded output.

For a broader discussion of what determinism means and why Flux requires it, see Concepts: determinism. For the implications on tasks that write to external systems, see Idempotency.

Debugging via replay

Replay lets you observe a failed execution without repeating its side effects. A standard debug sequence:

1. Inspect the failed execution:

   flux execution show --detailed <execution_id>

2. Identify the failing task. The event log will show a TASK_FAILED entry with the exception message.

3. Fix the task implementation.

4. Re-register the workflow:

   flux workflow register my_workflow.py

5. Replay the original execution. Tasks that completed before the failure will replay from the log without making API calls or database writes. Only the previously-failing task (and anything that follows it) will execute against live systems:

   ctx = my_workflow.run(execution_id=original_execution_id)

This narrows the blast radius during debugging: work that already succeeded is guaranteed not to repeat.

Tasks that may run again on replay

Replay skips completed tasks. Tasks that had not yet started when the original execution stopped run normally, and tasks that started but didn’t finish (no TASK_COMPLETED event written) re-run from the top. Any task interrupted between its function return and its checkpoint is a candidate for re-execution.

Designing tasks to be safe under re-execution is what makes the end-to-end behavior exactly-once from the perspective of external systems. See Idempotency for the two patterns that cover almost every case.

For the full event schema and the CLI reference, see Reference: flux execution show.