Running the server
Starting, supervising, and gracefully shutting down a Flux server in production — and what to do when it won't start.
You have flux-core installed and a database reachable from that host. This page is the next step: starting the server, keeping it running, and recovering when it doesn’t.
The server is a FastAPI process (flux/server.py plus the route modules in flux/api/) wrapping uvicorn. The scheduler runs inside it. Multiple server replicas are supported on PostgreSQL — replicas coordinate through the database, and the scheduler runs as a fleet-wide singleton via a PostgreSQL advisory lock, so schedules never dispatch twice. There is no leader election to configure. See High availability for the multi-replica topology and Schedule management for schedule semantics.
The command
flux start server
That boots uvicorn, mounts the FastAPI app, starts the in-process scheduler, and begins the heartbeat reaper that evicts stale workers. Two optional flags:
flux start server --host 0.0.0.0 --port 8000
Defaults resolve from configuration (server_host = localhost, server_port = 8000). For anything beyond local dev, set --host so the process binds to a routable interface.
Everything else — database URL, auth, encryption keys — comes from configuration sources in this order: environment variables, then flux.toml, then [tool.flux] in pyproject.toml, then defaults.
Environment variables
The FLUX_ prefix and __ nested delimiter come from pydantic-settings (flux/config.py). The ones you will set most often:
| Variable | What it controls |
|---|---|
FLUX_DATABASE_URL | SQLAlchemy URL. Default sqlite:///.flux/flux.db. For production use postgresql://user:pass@host:5432/flux. |
FLUX_HOME | Flux home directory (default .flux). The bootstrap-token file is persisted here. |
FLUX_SECURITY__AUTH__OIDC__ENABLED | Set to true to require OIDC. See Authentication overview. |
FLUX_SECURITY__AUTH__API_KEYS__ENABLED | Set to true to accept API-key auth. |
FLUX_SECURITY__ENCRYPTION__ENCRYPTION_KEY | Required for the secrets store and payload signing. No default in 0.56.0; with auth enabled the server refuses to start without it. |
FLUX_SECURITY__EXECUTION_TOKEN_SECRET | Signs per-execution JWTs. Required when auth is enabled — the server refuses to start without it. |
FLUX_DATABASE_URL supports ${VAR} interpolation, so postgresql://flux:${PG_PASSWORD}@db:5432/flux works.
If FLUX_WORKERS__BOOTSTRAP_TOKEN is unset the server generates one on first startup and writes it to <FLUX_HOME>/bootstrap-token (mode 0600). Retrieve it with flux server bootstrap-token.
TLS
flux start server does not expose --ssl-keyfile or --ssl-certfile. Terminate TLS at a reverse proxy and forward to Flux over plain HTTP on the loopback or a private network. A minimal nginx fragment:
server {
listen 443 ssl;
server_name flux.example.com;
ssl_certificate /etc/ssl/flux.crt;
ssl_certificate_key /etc/ssl/flux.key;
location / {
proxy_pass http://127.0.0.1:8000;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# SSE dispatch and progress endpoints stream — disable buffering.
proxy_buffering off;
proxy_read_timeout 1h;
}
}
The proxy_buffering off and long read timeout matter. Workers hold GET /workers/{name}/connect open as an SSE stream; an aggressive proxy timeout severs it and forces reconnects.
Supervision
The process is foreground-by-default; wrap it in your init system.
systemd
# /etc/systemd/system/flux-server.service
[Unit]
Description=Flux server
After=network-online.target postgresql.service
Wants=network-online.target
[Service]
Type=simple
User=flux
Group=flux
WorkingDirectory=/var/lib/flux
EnvironmentFile=/etc/flux/server.env
ExecStart=/usr/local/bin/flux start server --host 0.0.0.0 --port 8000
Restart=on-failure
RestartSec=5s
KillSignal=SIGTERM
TimeoutStopSec=30s
[Install]
WantedBy=multi-user.target
/etc/flux/server.env holds FLUX_DATABASE_URL, FLUX_SECURITY__ENCRYPTION__ENCRYPTION_KEY, and the auth toggles. chmod 600 it.
Docker
The image entrypoint runs flux start server. Mount FLUX_HOME as a volume so the persisted bootstrap-token survives restarts:
docker run -d --name flux-server \
-p 8000:8000 \
-e FLUX_DATABASE_URL="postgresql://flux:secret@db:5432/flux" \
-e FLUX_SECURITY__AUTH__API_KEYS__ENABLED=true \
-e FLUX_SECURITY__ENCRYPTION__ENCRYPTION_KEY="$(cat /etc/flux/encryption-key)" \
-v flux-home:/root/.flux \
ghcr.io/edurdias/flux:0.56.0
Kubernetes
Use a Deployment. Multiple replicas are supported when the database is PostgreSQL — schedule dispatch and retention sweeps are fleet-wide singletons coordinated through advisory locks, so a second replica does not double-fire schedules. Give worker SSE connections (/workers/{name}/connect) session affinity at the load balancer, and run flux db upgrade once before rolling replicas across a version boundary. The full multi-replica story is in High availability. On SQLite, keep replicas: 1 — SQLite is single-node only.
Either way, pin the bootstrap token explicitly (FLUX_WORKERS__BOOTSTRAP_TOKEN from a Secret) or mount a shared PersistentVolumeClaim at FLUX_HOME; a per-pod regenerated token will lock out every existing worker until they re-register. Workers run as their own Deployment and scale independently — see Running workers.
Graceful shutdown
Send SIGTERM. Uvicorn drains in-flight HTTP requests and closes the SSE worker connections; the scheduler and heartbeat reaper tasks stop in the FastAPI lifespan shutdown hook (flux/server.py::_create_api).
Workers do not lose state when the server goes away. Each worker reconnects with exponential backoff (capped at workers.reconnect_max_delay, default 60 seconds). In-flight executions on workers keep running during the outage — workers buffer checkpoint POSTs and retry them once the server is back.
The auth-disabled warning
On every startup with auth disabled, you will see:
CRITICAL - Authentication is DISABLED. All requests are treated as the ANONYMOUS
admin principal. This is not safe for production. Enable an auth provider via
[flux.security.auth.oidc] or [flux.security.auth.api_keys] before exposing this
server.
auth.enabled is a real, settable field (FLUX_SECURITY__AUTH__ENABLED). Enabling either oidc.enabled or api_keys.enabled also implies auth.enabled = true, and the server rejects auth.enabled = true with no provider configured at startup (flux/security/config.py). A production deployment must turn on at least one provider.
What can go wrong
Four things break first when the server won’t start.
Incomplete security configuration
Symptom. Startup raises RuntimeError: Refusing to start with incomplete security configuration.
Cause. With auth enabled, the server validates at startup that FLUX_SECURITY__EXECUTION_TOKEN_SECRET and FLUX_SECURITY__ENCRYPTION__ENCRYPTION_KEY are set — previously these surfaced only mid-traffic, when the first worker token was minted or the first secret stored. Debug mode is exempt.
Fix. Set both values (random 32+ bytes each) in the server environment. See Authentication overview.
Bind conflict
Symptom. Process exits with OSError: [Errno 98] Address already in use.
Fix. Find the owner with lsof -i :8000 or ss -ltnp 'sport = :8000'. Stop it, or pick a free port (flux start server --port 8001, or set FLUX_SERVER_PORT=8001).
Missing or unreachable database
Symptom. Startup fails before the Flux server started successfully log line. SQLAlchemy raises OperationalError (or psycopg.OperationalError). Schema migrations run on first connect (Alembic, flux/migrations/) — if the connection itself fails, the server exits.
Fix. Verify the URL from the server host: psql "$FLUX_DATABASE_URL" -c 'SELECT 1'. Common causes are a wrong password, a firewall rejecting the server’s IP, or the DB user lacking DDL permission on the target database (Flux applies migrations on first connect, so the user needs owner-level permissions — or run flux db upgrade once with a privileged DSN).
Bootstrap token state missing
Symptom. Workers fail registration with 401 Unauthorized on POST /workers/register after a server redeploy. The server logs nothing — it just doesn’t recognize the worker’s token.
Cause. The server resolves its bootstrap token from <FLUX_HOME>/bootstrap-token at first request (resolve_or_generate in flux/security/bootstrap_token.py). If FLUX_HOME is on ephemeral storage, the file is lost on redeploy, the server generates a new token, and the old workers’ tokens stop working.
Fix. Persist FLUX_HOME on durable storage (volume mount, PVC), or pin the token explicitly via FLUX_WORKERS__BOOTSTRAP_TOKEN. A configured value wins over the persisted file, so distribute the same value to every worker. With the override set, flux server bootstrap-token --rotate is a no-op until you remove it.
Next
- Server configuration — the full
flux.tomlsurface and everyFLUX_*variable that matters. - Authentication overview — turn on OIDC or API keys before exposing the port.
- Running workers — the other half of a working deployment.
- System architecture — what the server is and how clients, workers, and storage relate to it.