Skip to content

test: connect one CPU offline read, record, and assessment - #378

Merged
zengchuang-hw merged 1 commit into
mainfrom
cursor/cpu-offline-real-ports-eedc
Nov 10, 2026
Merged

zengchuang-hw merged 1 commit into
mainfrom
cursor/cpu-offline-real-ports-eedc

Conversation

@Dong1017

Copy link
Copy Markdown
Collaborator

Governing Issue and delivery slice

Refs #372
Refs #362

  • Accepted scope / delegation: After docs: record the target module layout and import gate #376 and test: add a fake prepare-to-assess handoff loop #377, the tracker's next suggested slice is the first real Knowledge read, one Harness CPU/offline record, and one Benchmark offline assessment. This PR follows that row. It does not treat the 3+1 split as a newly accepted plan.
  • What docs: record the target module layout and import gate #376 already delivered: The directory rule and the direct-import gate are on main at 5999ced639a88d359dfd7bb1798fef9548717eb5.
  • What test: add a fake prepare-to-assess handoff loop #377 already delivered: The test-local prepare, archive, bind, execute, and offline assess handoff is on main at f6d200775d554d08fb73673833611dd6ac02d14b. Those ports were injected fakes.
  • What this PR delivers: tests/architecture_real_ports.py is injected into that same handoff for the success connection. It calls KnowledgeReader.read on the installed package card transformers.common.llm.kv_cache_strategy_contracts (record.json only), execute_trial for one local Python process with no device, and evaluate on the retained record. The prepared bytes carry the knowledge revision, the harness fact carries the prepared digest, and the assessment bytes carry the same revision plus the process return code. The coordinator in tests/architecture_closed_loop.py still does not import the three modules.
  • What it does not deliver: No production Apps package, no empty knowledge/, benchmark/, or apps/ tree, no Bundle revision change, no collect or search_knowledge choice, no archive-owner or RunStore choice, no new exit-code table, no CLI or runner edit, no layout migration, and no corpus or fixture move. Refusal cases still inject outcomes the single CPU record does not produce. Issues 372 and 362 remain open.

Knowledge authorship declaration

Not applicable. This PR does not change knowledge content or Bundle revision rules. The test-local JSON digest is not inferrsi-json-v1. The read uses the existing package card and returns its current revision; that revision is not pinned in the tree.

{
  "schema_version": 1,
  "head_sha": "a0a2d81082d44e0fb8e1b7561d2a39a7db657398",
  "human_only": false,
  "material_ai_authors": [
    {
      "tool": "Cursor",
      "model_id": "grok-4.7",
      "effort": "unknown",
      "role": "author"
    }
  ]
}

Changed artifacts and evidence

Requirement Actual change Check / evidence and result Remaining gap
Real Knowledge read PackageKnowledgeRead.deliver calls KnowledgeReader.read The connection test read transformers.common.llm.kv_cache_strategy_contracts. A second read returned the same sha256: revision. Bundle content was record.json only. Passed Collection versus search_knowledge stays unset
One Harness CPU/offline record CpuOfflineHarnessRecord.execute calls execute_trial One local sys.executable process wrote trial/result.json (probe=cpu-offline, device=null) and trial/execution.json (status=completed, returncode=0, pid present, elapsed_seconds > 0). No device requested. Passed Attempt id cpu-offline-1 is allocated by the test port. runner.py is untouched because #367 is open
One Benchmark offline assessment OfflineBenchmarkAssessment.assess calls evaluate Verdict INCONCLUSIVE. improvement_pct is null. statistical_significance is not established. Reasons include the missing validation command and cpu offline record has no source snapshot. Assess patched subprocess.run and Popen and did not start a process. Passed A single record is not a paired measurement, so this is not a campaign PASS
Three refs connect Same run_loop prepared stores the knowledge revision. The record fact stores that prepared digest. The assessment stores the same revision and record_returncode=0. The record digest is the fact hash, and it is absent from the prepared archive. Passed Archive remains the test MemoryArchive. RunStore ownership stays unset
Import gate Existing gate test PYTHONPATH=src python3 -m unittest tests.test_architecture_closed_loop tests.test_target_module_import_gate: 19 passed in 0.397s A clean scan is not module isolation
No new core imports AST check in the test Ports import knowledge_operations, harness.scenarios.external, and evaluation. They do not import cli, arguments, harness.runner, or the absent knowledge, benchmark, or apps packages. knowledge_operations.py and external.py do not import each other. Passed evaluation.py still re-exports harness.evaluation. That existing edge is unchanged
Style Touched Python files python3 -m ruff check on the three Python files passed (ruff 0.17.4). git diff --check clean Full ruff check src tests tools and full pytest were not run

Checks and dependencies

  • Checks actually run (result and tested scope): PYTHONPATH=src python3 -m unittest tests.test_architecture_closed_loop tests.test_target_module_import_gate on CPU, with no device: 19 passed (15 closed-loop, including the real connection, and 4 import-gate) in 0.397s. Ruff 0.17.4 passed on tests/architecture_closed_loop.py, tests/architecture_real_ports.py, and tests/test_architecture_closed_loop.py. git diff --check was clean.
  • Failed, skipped, timed-out or not-run checks (if applicable): The unittest run had no failed, skipped, or timed-out tests. Full pytest, ruff check src tests tools, Ready CI, hardware, GPU/NPU, and model runs were not run. No workflow file was edited.
  • Dependencies / shared interfaces / integration assumptions (if applicable): Open feat: support shared YAML task input #367 still owns cli.py, arguments.py, harness/runner.py, and pyproject.toml. This branch does not edit them. task: Apps 应用层详细设计与迁移计划(CLI / Python / Plugin) #371 still names the Apps package. The assembler stays under tests/.

Remaining work and Issue closure

  • Remaining Issue requirements / follow-up: Docs and fixture-path cleanup remain the later suggested slice. Production Apps assembly waits on task: Apps 应用层详细设计与迁移计划(CLI / Python / Plugin) #371. S2, S3, and S4 production work is not in this PR. Undecided items stay undecided: new envelope exit numbers, Collection versus search_knowledge, archive ownership versus RunStore, and the Apps directory name.
  • Goal-acceptance owner or accepted closure delegation: Not claimed.
  • Goal ready for acceptance? Not claimed

Handoff (only when an actual handoff or decision changed)

Open in Web Open in Cursor 

The closed-loop coordinator still injects its ports. The success path now
calls KnowledgeReader.read, one local execute_trial, and offline evaluate.
Refusal cases still inject outcomes. Bundle revision rules are unchanged.

Agent-assisted-by: Cursor Grok 4.7 (implementer)
Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
Co-authored-by: Dong1017 <Dong1017@users.noreply.github.com>
@cursor

cursor Bot commented Nov 10, 2026

Copy link
Copy Markdown

@zengchuang-hw @amy-why-3459

Please review head a0a2d81082d44e0fb8e1b7561d2a39a7db657398.

This slice keeps the #377 test-local handoff and swaps the success-path ports for one real KnowledgeReader.read (transformers.common.llm.kv_cache_strategy_contracts, record.json only), one CPU/offline execute_trial record (local Python, device=null, returncode=0), and one offline evaluate. The observed verdict was INCONCLUSIVE: the process exit stayed an execution fact, with no paired measurement and no source snapshot. Issues 372 and 362 stay open.

Checks I ran on that head: PYTHONPATH=src python3 -m unittest tests.test_architecture_closed_loop tests.test_target_module_import_gate — 19 passed in 0.397s, no failures, skips, or timeouts. Ruff 0.17.4 passed on the three touched Python files. git diff --check was clean. Full pytest, ruff check src tests tools, hardware, and model runs were not run. Ready CI is in progress on GitHub and is not an acceptance result.

@Dong1017 Dong1017 added area:harness Harness module: profile, A/B loop, quality gates, evidence reports kind:task Bounded delivery task with a delegated slice labels Nov 10, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS
Reviewed base f6d2007 and head a0a2d81 (full diff on opened; no prior review).

Verified:

  • tests/architecture_closed_loop.py stays a port-injected handoff. The code change is docstring-only. test_coordinator_does_not_import_production_cores still rejects any inferrsi* import in that module.
  • tests/architecture_real_ports.py is the success-path assembler and remains under tests/. It calls KnowledgeReader.read on transformers.common.llm.kv_cache_strategy_contracts, execute_trial for one local probe, and evaluate once. test_real_ports_stay_on_three_existing_seams requires those three call names and rejects collect, search_knowledge, and inferrsi.cli / arguments / harness.runner / apps / knowledge / benchmark.
  • The read is gated on a bundle whose content key is only record.json. The harness keeps return code 0 and device: null as execution facts. OfflineBenchmarkAssessment calls evaluate with a null metric, an empty baseline, no quality command, and verified: False. test_real_cpu_offline_ports_connect_three_refs requires verdict INCONCLUSIVE, with the snapshot reason and the missing quality-command reason retained on the archived assessment.
  • The record digest stays out of archive.prepared. Prepared bytes and the assessment digest are what MemoryArchive is asserted to hold. docs/module-layout.md matches that boundary and leaves #371 as the Apps proposal. The card revision is read through; Bundle revision rules are unchanged.
  • The change is a connection test. min_improvement_pct is only an input on that inconclusive evaluate call.

Not verified here:

  • Tests were not executed, and CI was not inspected.
  • Bodies of KnowledgeReader.read, execute_trial, and evaluate were not re-opened beyond these call sites.

Optional follow-up (non-blocking): OfflineBenchmarkAssessment.assess installs a unittest.mock tripwire on subprocess.run and Popen around evaluate. That guards this assembler. A local from subprocess import run binding inside evaluate would slip past that patch.

@amy-why-3459 amy-why-3459 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS
Reviewed base f6d2007 and head a0a2d81 (new PR, full diff).

Verified:

  • The tests/architecture_real_ports.py calls match the current signatures at this head: KnowledgeReader.read(reference), execute_trial(command, cwd, trial_dir, deadline, metric_key) with metric_key=None (so no metric is read and only execution.json is written), and evaluate(spec, baseline, candidate, validation, integrity).
  • The INCONCLUSIVE outcome follows from harness/evaluation.py itself. It gets an expected/recorded trial-count mismatch (candidate is empty), a missing metric, validation=None ("quality validation command was not supplied"), and integrity.verified=False (the supplied snapshot reason). improvement_pct stays None, so the single CPU record cannot be read as a PASS.
  • tests/architecture_closed_loop.py only has docstring changes, and the coordinator still imports no inferrsi* modules. The new AST test pins the ports to the three existing seams and rejects cli/arguments/harness.runner/apps/knowledge/benchmark, collect, and search_knowledge.
  • docs/module-layout.md is consistent with the code. MemoryArchive holds only prepared and assessment bytes, and the record digest is not in archive.prepared.
  • All 7 check runs on this head passed: declarations, pr-references, full-check 3.10/3.11, installed-package 3.10/3.11, lint-and-smoke.

Not verified: I did not run the tests locally. I relied on the CI results above.

Optional (non-blocking): "exit_code": record["returncode"] in the evidence duplicates returncode. If nothing downstream consumes it, it could be dropped to avoid confusion with the envelope exit-code table, which is still undecided.

@zengchuang-hw zengchuang-hw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS
Reviewed base f6d2007 and head a0a2d81.

Delegation: zengchuang-hw directly authorized this OpenAI assistant to act as an independent reviewer for PR #378, reply with the review result, and submit Approve on PASS. This review is submitted under zengchuang-hw; it does not authorize integration or Issue acceptance.

No blocking findings in this bounded test/documentation slice.

  • Traced the real KnowledgeReader.read, execute_trial, and evaluate paths. The coordinator remains port-injected; the retained record binds the prepared digest, and the assessment retains the knowledge revision.
  • The local CPU process return code remains an execution fact. Missing paired measurements, validation, and source integrity correctly preserve INCONCLUSIVE.
  • Local exact-blob snapshot: PYTHONPATH=src python3 -m unittest tests.test_architecture_closed_loop tests.test_target_module_import_gate passed all 19 tests on Python 3.12.14.
  • Rechecked all 7 CI jobs and knowledge-model-policy: successful. Ready CI full-check on Python 3.10/3.11 passed; logs show 3560 passed, 14 skipped and 286 subtests passed per version, with Ruff passing. The tested merge tree matches this head.

Limits: the local snapshot was focused; full pytest and Ruff were verified through CI, not rerun locally. MemoryArchive and refusal-path injected outcomes remain test substitutes. No production Apps assembly, persistence/restart replay, full module isolation, hardware/model execution, or campaign acceptance was validated. Issues 372 and 362 remain open.

CI: https://github.com/InferMatrixLab/InferRSI/actions/runs/42743431583

@zengchuang-hw
zengchuang-hw merged commit b8df371 into main Nov 10, 2026
8 checks passed
@Dong1017
Dong1017 deleted the cursor/cpu-offline-real-ports-eedc branch November 10, 2026 03:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:harness Harness module: profile, A/B loop, quality gates, evidence reports kind:task Bounded delivery task with a delegated slice

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants