Google Gemini

Using Google Gemini models with Flux — setup, API keys, model strings, structured output, and thinking budgets.

Flux talks to Gemini through the google-genai Python SDK. The integration lives in flux/tasks/ai/gemini.py and uses the same agent() factory as every other provider.

Install

The google-genai SDK is an optional dependency. Install it directly:

pip install google-genai

Or take the full LLM bundle:

pip install flux-core[ai]

The ai extra pulls google-genai along with ollama, openai, and anthropic. There is no per-provider extra in 0.56.0.

API key

Flux instantiates genai.Client() with no arguments, so the SDK’s default credential lookup applies. The google-genai SDK accepts either GEMINI_API_KEY or GOOGLE_API_KEY:

export GEMINI_API_KEY="..."
# or
export GOOGLE_API_KEY="..."

If both are set, GEMINI_API_KEY wins. Application Default Credentials (Vertex AI) require additional configuration that the Flux integration doesn’t expose currently — use the API-key flow for the supported path.

Model string

The provider prefix is google/ (not gemini/). The model identifier follows the slash:

from flux.tasks.ai import agent

assistant = await agent(
    "You are a helpful assistant.",
    model="google/gemini-2.5-flash",
)

The integration is tested against gemini-2.5-pro and gemini-2.5-flash. Earlier gemini-1.5-* models and newer 2.x variants work the same way as long as they’re available to your API key.

Streaming

Streaming is on by default. The Gemini formatter uses client.aio.models.generate_content_stream() and yields text chunks through Flux’s progress() task.

Streaming is disabled automatically when response_format is set, because Gemini’s structured-output mode returns a single JSON document rather than incremental tokens.

max_output_tokens

Gemini honors a maximum output length, but the parameter name in the SDK is max_output_tokens, not max_tokens. Flux translates: the max_tokens argument on agent() is forwarded to the Gemini GenerateContentConfig as max_output_tokens (gemini.py:158):

assistant = await agent(
    "You are a helpful assistant.",
    model="google/gemini-2.5-flash",
    max_tokens=8192,  # becomes max_output_tokens=8192 in the API call
)

Default is 4096. Raise it for long outputs. Gemini and Anthropic are the two providers where max_tokens is actually honored — OpenAI and Ollama ignore it.

Reasoning effort (thinking budget)

Gemini 2.5 models support “thinking” — extra reasoning tokens spent before the visible response. Flux maps reasoning_effort to a token budget through ThinkingConfig:

budget_map = {"low": 1024, "medium": 4096, "high": 16384}

So reasoning_effort="low" allocates up to 1024 thinking tokens, "medium" up to 4096, and "high" up to 16384. The mapping lives in gemini.py:164.

researcher = await agent(
    "You are a careful researcher.",
    model="google/gemini-2.5-pro",
    reasoning_effort="high",
)

Thinking tokens are billed separately from output tokens — see Gemini’s pricing for the current rate. The Gemini formatter sets supports_reasoning_stream = True and surfaces thinking-deltas (part.thought = True) through the same agent-loop callbacks as text tokens.

Structured output

Gemini structured output uses the native response_mime_type="application/json" + response_schema configuration. The Flux formatter wires it like this (gemini.py:161-162):

config_kwargs["response_mime_type"] = "application/json"
config_kwargs["response_schema"] = self._response_format

response_schema is the Pydantic class itself — the google-genai SDK accepts Pydantic models directly and handles schema conversion.

from pydantic import BaseModel

class Recipe(BaseModel):
    title: str
    ingredients: list[str]
    steps: list[str]

generator = await agent(
    "Generate recipes from a description.",
    model="google/gemini-2.5-pro",
    response_format=Recipe,
)
result: Recipe = await generator("A simple tomato pasta.")

This is enforced at the API level. The Flux agent loop still calls model_validate_json() on the response for an extra Pydantic-level check.

The formatter takes one quiet precaution: when stripping tools from the call kwargs for the final no-tool turn (remove_tools_from_kwargs, gemini.py:214-234), it preserves response_mime_type, response_schema, and thinking_config. Without that, the post-tool-loop turn would silently drop structured-output settings and return freeform text.

What can go wrong

See also