Summary

Severity: Medium
Area: Documentation
Affected files: README.md:24, docs/ARCHITECTURE.md:115, docs/AUDIT.md:33, docs/DECISIONS.md:87-88, docs/PROGRESS.md:54,354, docs/PLAN.md:155
Suggested branch: docs/correct-audit-claims
Suggested PR title: docs(audit): correct false guarantees in README, architecture and ADR docs

This is finding AUDIT-16 of 18 from a consolidated audit of agentkit/. It is deliberately self-contained so it can be fixed on its own branch with its own PR.

Related findings: Several of the corrected statements only become true once AUDIT-01, AUDIT-04 and AUDIT-05 land; sequence accordingly.

AUDIT-16 — Medium — Documentation asserts guarantees the code does not provide

Files / lines

• README.md:24 — "sql_readonly: Read-only SQL executor with single-SELECT AST validation…" → the implementation is regex/keyword based (no AST) and is bypassable (AUDIT-01).
• docs/ARCHITECTURE.md:115 — "…DB role is read-only." → the tool connects as the app superuser.
• docs/AUDIT.md:33 — Read-Only SQL Tool verdict PASS ("AST/regex validation blocking DDL/DML"); also the Gemini Adapter row claims "thought signature preservation PASS" (AUDIT-04/05/06).
• docs/DECISIONS.md:87 — ADR-006: "m.model_dump(mode="json") so raw bytes fields are transparently base64-encoded" → false (pydantic defaults to UTF-8 decoding; non-UTF-8 bytes raise).
• docs/DECISIONS.md:88 — ADR-006: "Standardize the active default Gemini model to gemini-2.0-flash" → the code uses gemini-3.1-flash-lite everywhere (config.py:34-36, factory.py:41, gemini.py:21).
• docs/PROGRESS.md:54 and :354, docs/PLAN.md:155 — repeat the same gemini-2.0-flash and "safe base64 bytes encoding" claims.

Note: docs/DECISIONS.md:86 (ADR-006's role="tool") was already superseded by ADR-008 (docs/DECISIONS.md:113-125) — do not "fix" line 86; ADR-006 should instead be marked Superseded in part by ADR-008.

Why this is a bug

The docs are the contract used for review and for operator trust. Several statements assert controls that do not exist, and docs/AUDIT.md records PASS verdicts for components whose guarantees are broken — which is how these issues survived the previous remediation rounds.

Pass cases

• README.md:24 describes the actual mechanism (and the real read-only role once AUDIT-01 lands).
• docs/ARCHITECTURE.md:115 is true, or corrected to describe the real control.
• docs/AUDIT.md's component table is updated: the SQL row and the Gemini row no longer claim an unqualified PASS while the defects above are open.
• docs/DECISIONS.md ADR-006 is corrected (base64 + model id) and annotated as partially superseded by ADR-008; docs/PROGRESS.md:54,354 and docs/PLAN.md:155 are aligned with the code.
• The default Gemini model named in the docs matches Settings.LLM_MODEL / create_llm_client / GeminiClient (a test or a grep-based check can enforce this).
• A new ADR documents the SQL isolation model (dedicated role + AST validation) once implemented.

Definition of done (per AGENTS.md)

• New/updated unit and integration tests covering the pass cases above, all passing (pytest).
• ruff check ., ruff format --check . and mypy agentkit clean.
• Docstrings updated where behaviour changed; docs/PROGRESS.md updated (status, PR link, commit count, notes).
• A new ADR in docs/DECISIONS.md if an architectural decision was made.
• Branch pushed and PR opened (never commit directly to main).

Baseline at the time of writing: pytest is green on main (all tests pass; tests/real/* skipped unless AGENTKIT_REAL_INFRA=1).