Summary

Severity: Medium
Area: Resources / LLM clients
Affected files: agentkit/api/deps.py:137-160, agentkit/api/main.py:63-72
Suggested branch: fix/llm-client-lifecycle
Suggested PR title: fix(llm): share and close provider clients across requests

This is finding AUDIT-14 of 18 from a consolidated audit of agentkit/. It is deliberately self-contained so it can be fixed on its own branch with its own PR.

Related findings: Related: AUDIT-12 (settings for retries).

AUDIT-14 — Medium — LLM provider clients are created per request and never closed

Files

• agentkit/api/deps.py:137-160
• agentkit/api/main.py:63-72

Current code — agentkit/api/deps.py:149-160

    override: LLMClient | None = getattr(request.app.state, "llm_client", None)
    if override is not None:
        return override

    try:
        return create_llm_client_from_settings(settings)
    except Exception as exc:
        logger.error("Failed to initialize LLM provider %s: %s", settings.LLM_PROVIDER, exc)
        raise ServiceUnavailableError(
            "llm",
            f"LLM provider '{settings.LLM_PROVIDER}' could not be initialized.",
        ) from exc

Current code — agentkit/api/main.py:63-72

    try:
        yield
    finally:
        aclose_fn = getattr(redis_client, "aclose", None)
        if callable(aclose_fn):
            await aclose_fn()
        else:
            await redis_client.close()
        await engine.dispose()
        logger.info("Application lifespan torn down (DB & Redis resources released).")

Why this is a bug

Each request constructs a fresh provider client (openai.AsyncOpenAI / genai.Client), each owning its own httpx transport and connection pool. Nothing closes them: the lifespan only tears down Redis and the DB engine. Over time this leaks sockets/file descriptors and defeats connection pooling. Note: for Gemini, the synchronous Client.close() does not close the async transport — the async path is client.aio.aclose().

Pass cases

• The provider client is created once (application lifespan / app.state) and reused across requests.
• The client is closed on shutdown (await client.aio.aclose() for Gemini, await client.close() for OpenAI) — asserted by a test that inspects the closed flag of a stub client after the lifespan exits.
• An explicit app.state.llm_client override still wins and is still closed on shutdown if the app owns it (or is documented as caller-owned).
• A test asserts that N requests do not construct N provider clients (e.g. a construction counter).
• A test asserts the retry wrapper (RetryingLLMClient) still wraps the shared client and still applies the configured retry policy.

Definition of done (per AGENTS.md)

• New/updated unit and integration tests covering the pass cases above, all passing (pytest).
• ruff check ., ruff format --check . and mypy agentkit clean.
• Docstrings updated where behaviour changed; docs/PROGRESS.md updated (status, PR link, commit count, notes).
• A new ADR in docs/DECISIONS.md if an architectural decision was made.
• Branch pushed and PR opened (never commit directly to main).

Baseline at the time of writing: pytest is green on main (all tests pass; tests/real/* skipped unless AGENTKIT_REAL_INFRA=1).