Long-running research agents

Build agents that accumulate knowledge across days or weeks, survive worker restarts, and pick up mid-plan. Combines long-term memory, planning, pause/resume, and the harness to handle multi-session research work.

Some research jobs take longer than a single session. Surveying competitors or tracking a regulatory landscape across weeks means starting work, stopping, and picking the thread back up days later. A research agent has to survive that cycle without losing what it learned.

Four Flux primitives cover this. Long-term memory keeps facts across executions. Planning gives the agent a step structure it maintains and resumes. Pause/resume handles deliberate suspension between sessions. The agent harness wraps the whole thing as a YAML-defined agent with sessions the operator controls. The rest of this page builds a researcher that uses all four together.

What the researcher example ships

examples/agents/researcher.yaml is a ready-made harness-driven researcher agent:

name: researcher
model: anthropic/claude-sonnet-4-20250514
description: Research agent with persistent memory and skills
system_prompt: |
  You are a research assistant. Gather information from available
  tools, synthesize findings, and store key insights in long-term
  memory for future reference. Use skills when they match the task.
  Cite sources when possible.

tools:
  - system_tools:
      workspace: .
      timeout: 30

skills_dir: examples/ai/skills

long_term_memory:
  provider: sqlite
  connection: researcher_memory.db
  scope: "research:default"

planning: true
approve_plan: true
max_tool_calls: 20
max_tokens: 4096
stream: true

Three fields matter for multi-day operation. long_term_memory keeps findings in a SQLite file that survives worker restarts. planning: true injects six planning tools so the agent tracks progress across named steps. approve_plan: true pauses the workflow when the agent first proposes a plan, letting you review scope before it starts running tools.

This is a harness agent, not a Python workflow. You register it with flux agent create, start sessions with flux agent start, and resume them with flux agent session resume. Each session is a Flux workflow execution of agents/agent_chat, so everything that applies to workflow durability applies here too.

Running the example

flux agent create researcher --file examples/agents/researcher.yaml
flux agent start researcher --mode terminal

The agent starts a session and waits for your first message. Give it a research task:

> Research the current state of open-source LLM inference frameworks.
  Focus on throughput benchmarks and hardware requirements.

Because approve_plan: true, the agent will call create_plan with its proposed steps, then the execution pauses immediately. You see the pending plan in the session output, then Flux prints a session ID:

Session paused for plan approval.
Session ID: 7f3c2d1a-...

Review the plan with flux workflow status or inspect it in the terminal output. To approve and let the agent proceed:

flux workflow resume agents/agent_chat 7f3c2d1a-... '{}'

flux workflow resume takes three positional arguments: the workflow ref (namespace/name), the execution ID, and the JSON input to pass into the resume. An empty {} approves the plan as proposed. To modify the plan’s scope:

flux workflow resume agents/agent_chat 7f3c2d1a-... '{
  "steps": [
    {"name": "survey-frameworks", "description": "List major open-source LLM inference frameworks."},
    {"name": "benchmark-review", "description": "Find published throughput benchmarks.", "depends_on": ["survey-frameworks"]},
    {"name": "summarize", "description": "Write a summary with citations.", "depends_on": ["benchmark-review"]}
  ]
}'

Passing a steps key replaces the plan while preserving any steps already completed. The agent then works through each step using its system tools.

Stopping and resuming days later

Mid-session, you can end the terminal process with Ctrl+D or /quit. The session — the underlying Flux execution — stays in its current state. The agent is paused between tool calls.

Come back the next day and reattach:

flux agent session resume 7f3c2d1a-...

The agent continues from its last checkpoint in terminal mode. Facts stored in researcher_memory.db are still there. Plan state — which steps are done, which are pending — is preserved in the event log.

If the worker restarted while you were away, Flux replays the event log. Completed tool calls and plan steps come from the log. Only the next pending step runs against the LLM.

How the memory layer works

When long_term_memory is set in the harness YAML, four memory tools are injected automatically: store_memory, recall_memory, forget_memory, and list_memory_keys. The agent calls them on its own initiative.

The scope field namespaces the facts. "research:default" groups everything under one label. For an agent tracking multiple topics independently, you’d want a different scope per session, but the harness YAML doesn’t support per-session scope variation. If that matters, the in-workflow approach (below) handles it.

Facts live in researcher_memory.db on the worker’s filesystem. Back that file up or migrate it when moving to a new machine. The SQLite schema uses (agent, scope, key) as the primary key, so the same file can hold facts for multiple agents or scopes without collisions.

Building the same pattern as a Python workflow

If you need per-execution memory scopes, programmatic plan seeding, or tighter integration with other workflow steps, use the Python workflow directly. The harness is a thin wrapper around the same primitives:

from flux import workflow, ExecutionContext
from flux.tasks.ai import agent
from flux.tasks.ai.memory import long_term_memory, working_memory, sqlite
from flux.tasks.ai.tools.system_tools import system_tools


@workflow
async def research_agent(ctx: ExecutionContext[dict]):
    raw = ctx.input or {}
    topic = raw.get("topic", "general research")
    scope = raw.get("scope", "research:default")

    researcher = await agent(
        "You are a research assistant. Gather information from available tools, "
        "synthesize findings, and store key insights in long-term memory for "
        "future reference. Cite sources when possible.",
        model="anthropic/claude-sonnet-4-20250514",
        tools=system_tools(".", timeout=30),
        working_memory=working_memory(window=50),
        long_term_memory=long_term_memory(
            provider=sqlite("researcher_memory.db"),
            agent="research_agent",
            scope=scope,
        ),
        planning=True,
        approve_plan=True,
        max_tool_calls=20,
        max_tokens=4096,
    )

    return await researcher(f"Research: {topic}")

Pass a scope in the workflow input to keep topics isolated:

flux workflow run research_agent '{"topic": "LLM inference frameworks", "scope": "research:llm-inference"}'
flux workflow run research_agent '{"topic": "Vector database benchmarks", "scope": "research:vectordbs"}'

Each execution reads and writes only the facts scoped to its own label. The two agents can share the same researcher_memory.db file without interfering.

Explicit pause checkpoints

Some research tasks have natural stopping points: after gathering data, after a preliminary synthesis, before a write-up that needs human review. Use pause() from flux.tasks to insert those explicitly:

from flux import workflow, ExecutionContext
from flux.tasks import pause
from flux.tasks.ai import agent
from flux.tasks.ai.memory import long_term_memory, sqlite


@workflow
async def research_with_review(ctx: ExecutionContext[dict]):
    raw = ctx.input or {}
    topic = raw.get("topic", "general research")

    researcher = await agent(
        "Research the topic thoroughly. Store all key findings in memory. "
        "After completing your research, summarize what you found.",
        model="anthropic/claude-sonnet-4-20250514",
        long_term_memory=long_term_memory(
            provider=sqlite("researcher_memory.db"),
            agent="research_agent",
            scope=f"research:{topic.replace(' ', '-').lower()}",
        ),
        planning=True,
        max_tool_calls=30,
    )

    # Phase 1: gather and store findings
    findings_summary = await researcher(f"Research: {topic}")

    # Pause for human review before the write-up phase
    review_input = await pause("research-review", output={
        "summary": findings_summary,
        "message": "Review findings. Resume to continue to write-up phase.",
    })

    # Phase 2: produce the final output, optionally using reviewer feedback
    feedback = review_input.get("feedback", "") if review_input else ""
    prompt = f"Write a final research report on {topic}."
    if feedback:
        prompt += f" Incorporate this feedback: {feedback}"

    return await researcher(prompt)

The workflow suspends after Phase 1 finishes with state PAUSED. Resume with optional feedback:

# Resume with no changes
flux workflow resume research_with_review <execution_id> '{}'

# Resume with reviewer feedback
flux workflow resume research_with_review <execution_id> \
  '{"feedback": "Focus more on hardware requirements for consumer-grade GPUs."}'

pause() is durable. If a worker restarts after the resume but before the next checkpoint, Flux replays the event log. When it reaches the pause() call, it finds the recorded resume input and returns it instead of suspending again. Phase 2 runs exactly once.

What survives a worker restart

For both the harness and the Python workflow, the event log handles replay:

One caveat: a tool call that was in progress when the worker died will re-run. For idempotent tools (file reads, web searches) that is fine. For tools with side effects, make them idempotent or declare @task.with_options(requires_approval=True) to gate them — see Tool approval.

Configuration reference for multi-day agents

Several parameters are worth raising for agents that run for long periods:

# Harness YAML
planning: true
approve_plan: true          # review scope before work starts
max_tool_calls: 50          # planning tools count against this limit
max_plan_steps: 30          # how many plan steps the agent can create

long_term_memory:
  provider: sqlite           # or postgresql for multi-worker deployments
  connection: memory.db
  scope: "research:default"
# Python workflow equivalent
await agent(
    ...,
    planning=True,
    approve_plan=True,
    max_tool_calls=50,
    max_plan_steps=30,
    strict_dependencies=True,   # prevent plan steps from running out of order
)

strict_dependencies=True is worth setting when later steps genuinely cannot proceed without earlier ones — analysis needs data gathering to finish first. With the default False, the agent warns about unmet dependencies but proceeds anyway.

For deployments where multiple workers share the same memory database, use postgresql:

long_term_memory:
  provider: postgresql
  connection: postgresql://user:password@db-host/researchdb
  scope: "research:default"

Both backends use an upsert on (agent, scope, key), so concurrent writes from multiple workers on the same scope last-write-win rather than error.

What’s next