Skip to content

perf(engine): keep one Engine and compilation cache per server process - #16

Merged
MrScripty merged 10 commits into
mainfrom
perf/one-engine-per-server-process
Nov 12, 2026
Merged

MrScripty merged 10 commits into
mainfrom
perf/one-engine-per-server-process

Conversation

@MrScripty

Copy link
Copy Markdown
Owner

What this changes

One MCP server process now has one facade, one Engine and one compilation cache for its whole life. Before, it built a new facade and Engine for every tool call and threw them away afterwards.

Why

  • Building the facade cost about 3 ms of every call.
  • The compilation cache had to live in the server and be lent to each short-lived Engine. An Engine opened any other way (the command-line script, the tests) had no cache and recompiled a snapshot every time it was asked for it.

What stays the same

The store connection is still opened for each call and closed when the call ends. An idle server holds no store connection, no lock and no open store file, as before.

Before each call the Engine is readmitted. The store runs the same admission a new open of that file runs: application, version, exact schema, required profile, migration and due maintenance. So these are all observed exactly as when every call opened its own Engine:

  • another process's writes
  • a changed schema or version
  • a store file that was replaced, removed or overwritten

A transaction an operation did not end is rolled back when the connection closes, never committed.

When the Engine is reopened

The Engine is closed and a new one opened, which discards its cache, when:

  • an operation ends in a failure nothing classified, including one that application purpose reports as APPLICATION.INTERNAL_UNAVAILABLE
  • the call is interrupted
  • the repository root or store path resolves somewhere else
  • the store path names another file or none

In the last case the old file's audit record is also dropped, so a later file given the same inode number is audited.

Measurements

Live stdio server, observer on, fresh store, median of 50 calls:

Call Before After
Application read core with snapshot 20.1 ms 16.5 ms
Authoring inspect of a snapshot 4.59 ms 1.43 ms

Per-call store work is now 0.32 ms, against 3.76 ms for opening and closing a facade.

In the slowest test module, test_publication_recovery.py, compilations go from 226 to 131. All 95 repeat compilations of an already compiled snapshot are gone.

Behaviour differences to be aware of

  1. Reopening the Engine discards its cache, so the next call recompiles once. Before, the server's cache survived a failed call.
  2. A new stderr line is possible: Engine close failed: <Type>, if closing after a failure itself fails.
  3. MCPServer._compiled_cache is removed. Two retained measurement scripts under docs/plans read it and are not updated.
  4. An Engine always has a cache. StandardsEngine._compiled_cache is never None, and the constructor reads repository.root when no cache is lent.
  5. Each newly retained cache entry costs one accounting walk, about 9% of a compilation, even in a command-line call that never reuses it.

A design choice worth a look

A first version also kept the SQLite connection open between calls. That saved a further 0.1 to 0.25 ms a call, but an idle server then kept the store file open, and it needed extra rules to stay correct when another program overwrote the file. I dropped it. The original comment in mcp.py says per-call opening avoids keeping store state alive across idle sessions, and this version keeps that.

Verification

  • Local, on a quiet machine: all twelve package suites pass, the engine suite passes with 687 tests in 242 s on all cores, and verify.py --complete passes 24 of 24.
  • Before this change the same engine-suite run took 283 to 308 s.

Tests

  • New: test_engine_lifetime.py (16 tests) and 10 in the snapshot suite. They cover each reopen case above, rollback, another process's writes and schema changes, due maintenance under a writer, no descriptor or lock while idle, and a dispatch from a different thread.
  • Updated: tests that asserted a facade per call now assert one facade and one store connection per call.
  • Five tests that passed a stand-in object as the repository now pass a real GitRepository rooted in the test's temporary directory.
  • Eight single-line mutations of the new logic each fail at least one test.

Not in this pull request

docs/plans/storage-lifecycle/plan.md names "cross-operation connection lifetime" as a trigger to re-plan. This version does not keep a connection across operations, but it does keep the Engine. No plan record is added here.

Stacked on #15.

🤖 Generated with Claude Code

MrScripty and others added 10 commits November 12, 2026 07:20
…iliation

Six engine tests have failed on main since 22 October, when two changes
landed without the tests that depend on them being updated:

- Interface 48 (ad56443). Four publication tests asserted the literal
  "interface 47;" in the server instructions, and two contract examples
  still carried interface_version 47. The assertion now reads the number
  from the interface the server under test serves, so it cannot go stale
  again, and the examples say 48.
- Publication checkout reconciliation (8c5a4ad). The create-and-retire
  apply test still asserted that apply leaves the working tree untouched.
  It now asserts the documented behaviour: HEAD, index and worktree present
  the published candidate and no other local difference exists.

No production code changes. The suite-input manifest is regenerated; it was
already stale on main because 5bce311 added files without refreshing it.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The MCP server opens the snapshot store for each tool call, and every open
of an existing store ran PRAGMA integrity_check and foreign_key_check over
the whole file. On a 282 MB store that is about 0.6 s added to every
route, read and related call, growing with the file.

The full audit now runs at most once per process for each store file,
identified by device and inode. It still runs on the first open in every
process, throughout a version 1 migration, and for a file put in place
under a different inode. Application id, schema version and exact schema
checks still run on every open. Measured on a copy of that store, five
consecutive opens in one process go from about 0.6 s each to 0.6 s for the
first and under 1 ms for the rest.

Known limits, documented in the snapshots README: a one-shot CLI call is a
new process and still audits; a store deleted and recreated in place can
reuse its inode on ext4 and is then not audited again by a server that is
already running.

Tests cover one audit for two opens, a replaced file, a corrupt
replacement, the checks that stay on every open, a fresh process rejecting
a corrupted file, migration keeping all of its audits, and the server
auditing once across several tool calls.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The recorder exists to show how agents use the tool interface so the
interface can be improved. As built it could not do that: it was off
unless two flags were passed, so the application purpose was never
recorded; it gave up for the life of the process after a 20 ms lock wait,
so servers started together recorded nothing; and it stored argument
shapes rather than values, so it could not say which standard was read,
which facts were sent or which rejection came back.

Always on. Every server process records to
${XDG_DATA_HOME:-~/.local/share}/standards-engine/observations/
interactions-v2.sqlite3 with no flags. --no-observe or
STANDARDS_ENGINE_OBSERVE=off turns it off. In-process MCPServer
construction records only when given a configuration, and every test and
harness launch passes --no-observe; a guard test fails if a new launch
site forgets.

Invisible. Tool results are unchanged, a healthy recorder writes nothing
to stderr, and a recording failure never changes or fails a call.

No dropped records. A busy database queues the row in a bounded in-memory
backlog and delivers it at the next call or at close. The database uses
write-ahead mode. Eight processes started at the same instant on a new
database now keep all 176 starts and terminals; the previous code kept 42.
Loss that does occur (backlog overflow) is counted in the database.

Seven-day window. Unchanged. A full file now evicts its oldest records
instead of refusing new ones. Schema v2 uses a new file; a v1 file is
refused and left untouched.

Useful values. Navigation calls keep targets, options and route facts as
sent, bounded to 200 characters per string. Authoring calls keep only
identifiers and enum words, never bodies, titles, rationales or evidence.
Results keep kind, status, code and the selected or returned module IDs.
A client thread id is stored only as a keyed hash, with a main or
subagent marker. No other request metadata is read.

The report gains a readable text summary (rejection rates, latency and
size percentiles, refused fact names and values, most-read targets,
repeat reads per thread) and a --since-days filter.

MCP-OBSERVABILITY.md is rewritten for the new contract.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…labelled

Application approval was a gate: one module without a current approval
anywhere in a selection rejected the whole route with "awaiting
application publication", naming nothing. 40 of 85 registered materials
are not currently approved, including two of three application profiles,
and any edit to Core or the Router would have taken every application call
offline until re-approval. In recorded use, 43 of 91 application routes
were rejected this way and agents responded by deleting true facts from
their queries.

Approval is now a label. This is interface 49.

- Every registered material a call selects or names is delivered. Each
  read result and each reading-plan entry carries review_state: current,
  changed-since-review or not-reviewed. The approval manifest and the
  authoring-side exposure workflow are unchanged.
- An unregistered or misspelt identity is rejected as
  APPLICATION.UNKNOWN_TARGET, and a traversal a relationship group does
  not permit as APPLICATION.TRAVERSAL_NOT_PERMITTED.
  APPLICATION.CONTENT_UNAVAILABLE is removed.
- Application route content pages are sized for inline display: about
  40 KiB of structured result by default (--route-page-bytes, 8 KiB to
  2 MiB), always at least one whole module. The previous 2 MiB bound
  produced pages the client diverted to a file. Authoring paging is
  unchanged.
- The application connection instructions and tool descriptions are
  rewritten in plain language: what the server is for, the sequence of
  calls, snapshot reuse and what the labels mean. A new application skill,
  .agents/skills/coding-standards/SKILL.md, teaches the same sequence with
  a worked example taken from real engine output.
- Tests that hard-coded the interface number now read it from the
  installed interface.

The generated contract and tool catalog are regenerated by the repository
generator. The CI path filter gains the new skill folder.

One current guide, docs/guides/positive-guidance-and-provenance.md, still
describes the gate. It is pinned by a coverage certificate and is left for
a certificate retirement.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…anged

Every server start validated the contract schema against the JSON Schema
metaschema: about 0.36 s of a roughly one-second start, for a file that
is the same on every start. Real agent sessions paid it at every
connection and the test suite paid it hundreds of times.

The generator now records the digest of the schema it validated in the
generated contract (VALIDATED_SCHEMA_DIGEST). The engine passes that digest
to compile_contracts, which skips only the metaschema validation when the
schema it was given has the same digest. Every other check still runs on
every call.

- A schema edited without regenerating has a different digest and is fully
  validated, as before, so an invalid or drifted schema is still rejected
  with CONTRACT.INVALID_SCHEMA.
- The comparison is made inside compile_contracts against the schema in
  hand, rather than trusting a caller's flag.
- The digest is of the JSON value, so key order and formatting do not
  matter, but 1, 1.0 and true are distinct, and a value that does not read
  back equal (a tuple, a non-string key) has no digest and is validated.
- The generator, its --check mode and the contract_projection verifier
  check never pass a digest and always validate.

Startup therefore relies on the validation done at generation time. A
server answering only initialize goes from about 0.75 s to about 0.42 s on
the development machine.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ll cores

The Engine suite took about 28 minutes serially on the development machine
and about 44 minutes in CI. Three things made it slow: tests started a new
server process for every call, two scenarios were run twice, and the suite
ran on one core.

One server process. A shared helper, tests/stdio_server.py, starts the
real server once and exchanges newline-delimited JSON-RPC with it for as
many calls as a test makes. It checks stderr after every response and
requires a clean exit at close. The per-call launch helpers now use it. A
test starts a further process only where its claim is about a separate
process (a cold read after publication, completion or recovery in a fresh
process, a continuation honoured by a process that did not issue it, two
processes contending for the store, a launch-option check, the recorder's
simultaneous-start tests); each such place carries a comment naming the
claim. Server processes per suite run fall from 262 to 117.

No scenario twice. BatchedPublicationTests no longer inherits and reruns
the two publication walkthroughs; one focused test covers what was
particular to it (compact summaries, the inline first work page, paging
through workflow_details, mixed resolve_many batches, review reaching
ready). The test that reran the whole supporting-workflow scenario on warm
caches is merged into that scenario, which now runs once on per-purpose
caches and asserts both were hit.

Two things are no longer exercised: verify, apply and recover over MCP
after a batch-completed review (batch and serial decisions producing the
same authority is proved in process by test_review_workflow_ux), and the
supporting-workflow scenario on cache-less facades (cache-less publication
remains covered by test_agent_workflow, test_publication_recovery,
test_analysis and test_navigation_indexes).

All cores. tests/parallel_unittest.py runs one `python -m unittest` worker
per test module, as many at a time as there are cores. unittest still
discovers and runs every test. The run fails if a worker fails, or if the
workers together ran a different number of tests than one serial discovery
finds. CI uses it for the Engine suite.

Running in parallel exposed tests that created a snapshot store inside
the repository under test; they now use a temporary installation root.

Measured on the development machine: serial 1,667 s before, 1,379 s after;
283 s on all 24 cores; 448 s with --jobs 4. Test count 670 (two reruns and
one merged test removed, one focused test and four helper tests added).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…kept lists

Fifty-five metadata_graph checks each validated a hand-written list of module
paths. The lists named 57 of the 74 canonical modules between them, repeated
the same validation up to 55 times, and could not fail for a reason the
others would not also report. A module added to the corpus was covered only
if someone remembered to add it to a list.

Replace them with one check that reads the canonical corpus manifest and
validates every module it declares:

- metadata_graph gains a `corpus` mode naming the manifest.
- standards_metadata exposes canonical_module_corpus_members so the check
  and load_canonical_module_corpus share one reader.
- New suite canonical-corpus-metadata holds the single whole-corpus check;
  a verifier test fails if the repository registers more or fewer than one.

The removals were made by the Engine's maintain_evidence operation (preview,
then apply), not by hand: 55 checks retired, the 50 suites left empty and
their registry entries removed, five suite descriptions updated. No retained
suite required a removed one, and the policy-impact catalog, relationships
and coverage requirements are unchanged.

Kept on purpose: release-maintenance-policy/metadata-closure. That suite is
a node in the policy-impact catalog, so retiring it changes every coverage
requirement ID and turns the 16 current certificates stale. It needs an
owner decision.

Suites 73 -> 24, checks 121 -> 67.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… table

The three metadata_route checks compared columns of a scenario table with
other columns of the same table. The scenario facts were written in a
vocabulary the Router does not have, were never bound to Router facts, and
the Router was never asked. Changing a routing rule could not make them
fail, and neither could an empty table.

Each scenario is now bound to Router facts and decided by the Router:

- RouterProjection.decide(facts) returns a RouteDecision. The Engine's
  _bound_routing_selection calls it, so the Engine and the check share one
  implementation.
- A check declares `facts` (Router fact IDs used as columns) and no longer
  declares `base_modules`; the Router supplies its own base.
- Every header column must have exactly one role, so no column goes unread.
- The Router's outcome is compared with the expected-selection columns, the
  route state, and the route table's direct modules and Requires closure.
- A table with no scenario fails (ROUTING.NO_SCENARIOS). A column or value
  the Router does not declare fails.

The decision tables are rewritten in routing.* facts. Scenarios that the
Router's vocabulary cannot tell apart are merged, so the verifier-change
table goes from 9 scenarios to 5 (21 to 17 overall). The route tables are
updated to what the Router selects today: its own base module, the
topic.code-design detail that architecture selects, and the contract detail
modules that boundary rules select.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…fest is stale

verify.py --complete ran every suite and passed even when the committed
suite-input manifest no longer matched the repository. Per-file digests
caught a changed pinned file, but nothing caught a changed list of tracked
files, so adding, removing or renaming a file left the manifest stale with
all 24 suites green. The engine test suite noticed about fifteen minutes
later; the seven-second checkpoint did not.

- suite_input_projection_diagnostic compares the committed manifest
  byte-for-byte with a fresh compilation and explains a mismatch.
- run_complete_verification reports SUITE_INPUT.STALE_MANIFEST (exit 2)
  for the canonical registry. The message names the refresh command and
  says to stage files first, because the file list is read from the Git
  index. JSON output carries the error beside the suite results.
- The Engine's read-only verify_repository uses the same function and now
  reports a stale manifest; its refresh writes the manifest and then passes.
- test_evidence_maintenance tracked a file after refreshing the manifest,
  which is this defect; the order is fixed.
- README: the documented verify_repository command lacked --purpose
  authoring and failed as written.

The pull-request path filter is removed from the workflow. The manifest
pins the whole tracked file list, so a pull request that only adds a file
under an unfiltered path would skip CI and turn main red after merge.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
An MCP server built a new facade and Engine for every tool call and threw
them away afterwards. Building the facade cost about 3 ms of every call.
The compilation cache had to live in the server and be lent to each
short-lived Engine, so an Engine opened any other way (the command-line
script, the tests) had no cache and recompiled a snapshot each time it was
asked for it.

One server process now has one facade, one Engine and one compilation
cache, opened by its first store-backed call and closed with the server.
An Engine always has a cache: its own unless an owner lends one.

The store connection is still opened for each call and closed when the
call ends, so an idle server holds no store connection, lock or open store
file, as before. Before each call the Engine is readmitted: the store runs
the admission a new open of that file runs (application, version, exact
schema, required profile, migration, due maintenance), so another
process's writes, a changed schema and a replaced, removed or overwritten
store are observed exactly as when every call opened its own Engine.

The Engine is closed and reopened, discarding its cache, when:

- an operation ends in a failure nothing classified, including one that
  application purpose reports as APPLICATION.INTERNAL_UNAVAILABLE;
- the call is interrupted;
- the repository root or store path resolves somewhere else;
- the store path names another file or none. The old file's audit record
  is dropped, so a later file given the same inode number is audited.

A transaction an operation did not end is rolled back when the connection
closes, never committed.

MCPServer._compiled_cache is removed. Tests that asserted a facade per
call now assert one facade and one store connection per call.

Measured on a live stdio server with a fresh store, median of 50 calls:
application read 20.1 -> 16.5 ms, authoring inspect 4.59 -> 1.43 ms.
The slowest test module goes from 226 compilations to 131.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 12, 2026

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 98952556-60ef-4112-8561-5ed4740e3291
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@MrScripty
MrScripty merged commit b332195 into main Nov 12, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant