Bootstrap tokens

The shared secret that lets Flux workers register with the server — generation, distribution, rotation, and the failure modes when it's missing.

The bootstrap token is the shared secret a worker presents on its first call to POST /workers/register. The server compares it byte-for-byte with hmac.compare_digest; on a match the worker gets back a session_token (and, when API-key auth is on, a freshly-provisioned worker principal with its own API key) for every subsequent SSE connect, claim, and checkpoint. The bootstrap token itself is never sent again — until the worker’s API key expires (worker_key_ttl, default 7 days): on the first 401 after expiry the worker re-registers with the bootstrap token automatically and receives a fresh key, so key rotation needs no operator action but does require every worker to keep holding a valid bootstrap token for its lifetime.

Because the token is a fleet-wide join secret, the registration endpoint is rate-limited per client IP ([flux.workers] register_rate_limit, default "30/minute", slowapi syntax, "" disables) to blunt online brute force. Large fleets restarting behind one NAT should raise it — or run the server behind a proxy that forwards real client IPs.

Generating

On the server host:

flux server bootstrap-token            # print the active token
flux server bootstrap-token --rotate   # generate and persist a new one

The first time the server starts without FLUX_WORKERS__BOOTSTRAP_TOKEN or [flux.workers] bootstrap_token set, it auto-generates secrets.token_hex(32), writes it to <home>/bootstrap-token with mode 0600, and logs a WARNING line so you can grep the log. The CLI reads that file back.

Resolution order (flux/security/bootstrap_token.py::resolve_or_generate): env var or config value (whitespace-only treated as unset), then the persisted file, then auto-generate. The server caches the resolved token in memory at startup. It must persist across restarts — if the file is deleted with no env-var override set, the next start generates a fresh secret and every existing worker stops authenticating.

Distributing to workers

Workers never auto-generate. They read FLUX_WORKERS__BOOTSTRAP_TOKEN (or [flux.workers] bootstrap_token) and crash at startup if it is missing or whitespace-only:

RuntimeError: Worker bootstrap token is not configured. Set FLUX_WORKERS__BOOTSTRAP_TOKEN
or 'bootstrap_token' under [flux.workers] in flux.toml.

systemd:

[Service]
EnvironmentFile=/etc/flux/worker.env   # FLUX_WORKERS__BOOTSTRAP_TOKEN=...
ExecStart=/usr/local/bin/flux start worker

Docker:

docker run -e FLUX_WORKERS__BOOTSTRAP_TOKEN="$TOKEN" -e FLUX_MODE=worker your-registry/flux:0.56.0

Kubernetes (mount as an env var from a Secret):

env:
  - name: FLUX_WORKERS__BOOTSTRAP_TOKEN
    valueFrom:
      secretKeyRef:
        name: flux-bootstrap
        key: token

Storage on the server

The persisted token lives at <home>/bootstrap-token — a plain text file with mode 0600, owned by the user running the server process. There is no database column and no encryption-at-rest; the file is the source of truth when no env-var or config override is set. If file storage is unacceptable, set FLUX_WORKERS__BOOTSTRAP_TOKEN from a secret store at process start and the file is never read. If permissions drift, run chmod 600 "$FLUX_HOME/bootstrap-token".

Rotating without dropping every worker

There is no per-token revocation primitive — the server holds exactly one valid bootstrap token at a time, cached in memory at startup. Safe rotation needs an overlap window:

  1. flux server bootstrap-token --rotate — the file is updated; the running server is unaffected yet.
  2. Distribute the new value to every worker host (Secret / EnvironmentFile / Compose env).
  3. Restart the server. It re-reads the file and now accepts only the new token. Existing workers keep working — their session_token is independent of the bootstrap token.
  4. Restart the workers (rolling) so they pick up the new env var. Each one re-registers cleanly.

If an env-var or config-file override is set, --rotate warns that the file write will not change the active token until the override is removed. Plan rotations against whichever source you actually use.

What can go wrong