Choosing a provider

Flux routes every agent() call through one of four provider modules: anthropic, openai, google (Gemini), or ollama. Each module translates Flux’s shared agent loop into the wire format that provider’s SDK expects. The choice affects which capabilities are available, whether you need an API key, and how certain parameters behave.

The model string

Every agent() call takes a model parameter in "provider/model_name" format. Flux splits on the first / to select the provider; everything after is forwarded as the model identifier.

model="anthropic/claude-sonnet-4-20250514"
model="openai/gpt-4o"
model="google/gemini-2.5-flash"
model="ollama/qwen3:8b"

If the string contains no /, Flux raises a ValueError before contacting any API.

API key setup

Anthropic, OpenAI, and Gemini each require an API key. Set the appropriate environment variable before running your workflow:

# Anthropic
export ANTHROPIC_API_KEY="sk-ant-..."

# OpenAI
export OPENAI_API_KEY="sk-..."

# Google Gemini
export GOOGLE_API_KEY="..."

Ollama reads no API key. It connects to a locally running ollama serve process (default http://localhost:11434). Pull the model first:

ollama pull qwen3:8b
ollama serve          # keep this running

Capability matrix

The four providers share the same agent() interface but differ in how they implement specific features:

* Ollama tool support depends on the local model. Models without native function-calling support fall back to text-based tool extraction.

** Ollama structured output is mutually exclusive with tool use. When both response_format and tools are set, Ollama logs a warning and drops the format constraint, using tool calling only.

† Ollama maps the "low", "medium", and "high" levels to distinct Ollama think settings. Models that only support an on/off thinking toggle fall back to enabling thinking.

Streaming

All four formatters implement async def stream() and support reasoning streams (supports_reasoning_stream = True). Streaming is enabled by default and disabled automatically when response_format is set on any provider.

Tool calling formats

Each provider uses a different wire format for tools. Flux handles the conversion in each provider module:

• Anthropic — wraps each tool schema under "input_schema" (not "parameters").
• OpenAI and Ollama — wrap each tool under {"type": "function", "function": {"name": ..., "description": ..., "parameters": ...}}.
• Gemini — uses FunctionDeclaration objects with parameters_json_schema, batched into a single Tool object.

Ollama does not return stable tool-call IDs. Flux generates a call_{uuid} ID for each call so that working memory and replay remain consistent across multi-turn loops.

Structured output

The mechanism differs by provider.

OpenAI and Gemini activate a native JSON mode:

• OpenAI sets response_format={"type": "json_schema", "json_schema": {...}} in the API call.
• Gemini sets response_mime_type="application/json" and response_schema=<PydanticModel> in the generation config.

Anthropic enforces structured output at the API level. The Anthropic API has no response_format parameter, so Flux registers a synthetic tool (respond_with_structured_output) whose input_schema is the Pydantic model and sets tool_choice to force that tool. Anthropic must call the tool, and its arguments are schema-valid by construction. The forced tool call is mutually exclusive with caller-supplied tools, so Flux only applies it when no other tools are configured.

Ollama appends the schema to the last user message and sets format="json". When tools are also configured, Ollama logs a warning and skips the format constraint, using tool calling only.

reasoning_effort

Set reasoning_effort="low", "medium", or "high" on agent() to activate chain-of-thought thinking. Each provider maps this differently:

• Anthropic — sets thinking={"type": "adaptive"} and output_config={"effort": <value>}.
• OpenAI — passes reasoning_effort=<value> directly to the Chat Completions API.
• Gemini — converts to a token budget: low→1024, medium→4096, high→16384 tokens, set via ThinkingConfig.
• Ollama — passes the chosen level through as the think setting, preserving the "low", "medium", "high" distinction. Models that only support an on/off thinking toggle fall back to enabling thinking.

Reasoning effort increases latency and cost on cloud providers. On Ollama, it only takes effect on models with built-in thinking support (such as qwen3:8b).

max_tokens

The max_tokens parameter controls the maximum response length.

• Anthropic — passes it as max_tokens in each API call. Default is 4096.
• Gemini — passes it as max_output_tokens in the generation config. Default is 4096.
• OpenAI — passes it as max_completion_tokens in the Chat Completions call.
• Ollama — ignores max_tokens. The Ollama formatter sets no token limit.

Which provider to use

Ollama is the practical choice for local development. There is no API key to wrangle, no network dependency, and tokens are free. Pull a model once and iterate offline:

assistant = await agent(
    "You are a helpful assistant.",
    model="ollama/qwen3:8b",
)

The tradeoff: local models are slower and use more RAM, with capability that varies by model. Tool calling and structured output work on models that support them, but not universally.

Anthropic, OpenAI, and Gemini are appropriate for production workloads. Managed infrastructure gives them predictable latency under load and handles concurrent requests without saturating local hardware.

The recommended pattern for a Flux project is to default to Anthropic in production code samples (using anthropic/claude-sonnet-4-20250514 or the current equivalent) and switch to Ollama for local runs by changing the model string. The rest of the agent configuration stays identical across both:

import os
from flux import ExecutionContext, workflow
from flux.tasks.ai import agent

MODEL = os.getenv("FLUX_AGENT_MODEL", "ollama/qwen3:8b")

@workflow
async def my_agent_workflow(ctx: ExecutionContext):
    assistant = await agent(
        "You are a helpful assistant.",
        model=MODEL,
    )
    return await assistant(ctx.input["question"])

Set FLUX_AGENT_MODEL=anthropic/claude-sonnet-4-20250514 in production; leave it unset locally to use Ollama.

Installing provider SDKs

Each provider requires its own Python package. Install only what you need:

pip install anthropic      # Anthropic
pip install openai         # OpenAI
pip install google-genai   # Google Gemini
pip install ollama         # Ollama

Or install all AI providers at once with the ai extra:

pip install flux-core[ai]

If you call agent() with a provider whose package is not installed, Flux raises an ImportError with the exact pip install command needed.

What’s next

• Tools and structured output — giving the agent Flux tasks it can call as tools, and using response_format for typed JSON responses.
• Reasoning models — the reasoning_effort parameter in detail, including cost and latency tradeoffs per provider.
• Streaming responses — how token streaming works in the Flux agent loop.