Encryption at rest

Flux encrypts a narrow slice of its database: the secrets.value column, and nothing else. Workflow source, configs, event log values, principals, and schedules are stored in plaintext columns. For disk-level confidentiality of the rest, layer it on at the database (PostgreSQL TDE) or filesystem level.

What is encrypted

One column, in flux/models.py:

EncryptedType is a SQLAlchemy TypeDecorator. On write it dill.dumps the value, encrypts the bytes, and base64-encodes the result.

Things you might assume are encrypted but aren’t:

• Workflow source. Stored verbatim in workflows.source. Workers receive it base64-encoded over SSE — that’s transport encoding, not encryption.
• Configs. configs.value is plain TEXT. Treat the config store as readable to anyone with database access.
• Event log values. execution_events.value is a pickle blob, not encrypted.
• API keys. Stored as SHA-256 hashes in api_keys.key_hash. The plaintext is shown once at creation and never persisted; it’s a one-way hash, not encryption.
• Bootstrap and execution tokens. Held in memory and (for bootstrap) the <home>/bootstrap-token file at mode 0600. Not in the database.

Algorithm

AES-256 in GCM mode, via PyCryptodome (Crypto.Cipher.AES, AES.MODE_GCM). Each encrypt generates a fresh 32-byte salt and 16-byte nonce; the stored blob is salt || nonce || tag || ciphertext, base64-encoded.

Key derivation

PBKDF2-HMAC-SHA256, 1,000,000 iterations, deriving a 32-byte AES-256 key per write. The salt is per-row.

PBKDF2(
    password=self._get_key().encode("utf-8"),
    salt=salt,
    dkLen=32,
    count=1000000,
    hmac_hash_module=SHA256,
)

The “master key” you supply is therefore a passphrase, not a raw key. It gets .encode("utf-8")-ed and fed straight into PBKDF2 — Flux does not base64- or hex-decode it. The shipped flux.toml suggests openssl rand -hex 32 (a 64-character hex string = 32 bytes of entropy), but any high-entropy string works.

Master key sourcing

Set one of, in order of precedence (flux/config.py::EncryptionConfig):

1. Environment variable: FLUX_SECURITY__ENCRYPTION__ENCRYPTION_KEY (recommended for production).
2. flux.toml: [flux.security.encryption] encryption_key = "...".

There is no file-path option — Flux reads the literal string. If the key is unset, the first write to secrets.value raises ValueError("Encryption key is not set in the configuration."). Reads of existing rows fail the same way. The server itself starts fine without a key; only secret operations fail.

Rotation

Flux 0.56.0 does not ship an encryption-key rotation primitive. There is no flux secrets re-encrypt; the only rotate in the security module is for bootstrap tokens.

To rotate manually:

1. With the old key still set, flux secrets list and flux secrets get each secret.
2. Stop the server. Set FLUX_SECURITY__ENCRYPTION__ENCRYPTION_KEY to the new key.
3. Start the server, then flux secrets set each value again — that re-encrypts it under the new key.
4. Delete any backups of the plaintexts.

Each row carries its own salt, so there’s no bulk “decrypt with old, encrypt with new” path through Flux’s API. Round-trip through plaintext as above, or write a script that imports EncryptedType and swaps the configured key between rows.

Failure modes

• Key missing in production. Writes and reads on the secrets store raise ValueError. Workflows that declare secret_requests fail at task entry. Fix: set the env var and restart.
• Key changed without re-encrypting. Existing ciphertext is unreadable; process_result_value raises ValueError("Failed to decrypt value: ..."). Fix: restore the old key, re-encrypt as above, then swap. If the old key is gone, the secrets are gone — recreate them from source.
• Key leaked. Rotate immediately, then audit access to whatever secret store (Kubernetes Secret, Vault, SSM Parameter Store) was holding it. Assume any attacker with a database copy now has the plaintext secrets.