Repository navigation
test: connect one CPU offline read, record, and assessment - #378
Conversation
The closed-loop coordinator still injects its ports. The success path now calls KnowledgeReader.read, one local execute_trial, and offline evaluate. Refusal cases still inject outcomes. Bundle revision rules are unchanged. Agent-assisted-by: Cursor Grok 4.7 (implementer) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com> Co-authored-by: Dong1017 <Dong1017@users.noreply.github.com>
|
Please review head This slice keeps the #377 test-local handoff and swaps the success-path ports for one real Checks I ran on that head: |
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed base f6d2007 and head a0a2d81 (full diff on opened; no prior review).
Verified:
tests/architecture_closed_loop.pystays a port-injected handoff. The code change is docstring-only.test_coordinator_does_not_import_production_coresstill rejects anyinferrsi*import in that module.tests/architecture_real_ports.pyis the success-path assembler and remains undertests/. It callsKnowledgeReader.readontransformers.common.llm.kv_cache_strategy_contracts,execute_trialfor one local probe, andevaluateonce.test_real_ports_stay_on_three_existing_seamsrequires those three call names and rejectscollect,search_knowledge, andinferrsi.cli/arguments/harness.runner/apps/knowledge/benchmark.- The read is gated on a bundle whose content key is only
record.json. The harness keeps return code 0 anddevice: nullas execution facts.OfflineBenchmarkAssessmentcallsevaluatewith a null metric, an empty baseline, no quality command, andverified: False.test_real_cpu_offline_ports_connect_three_refsrequires verdictINCONCLUSIVE, with the snapshot reason and the missing quality-command reason retained on the archived assessment. - The record digest stays out of
archive.prepared. Prepared bytes and the assessment digest are whatMemoryArchiveis asserted to hold.docs/module-layout.mdmatches that boundary and leaves #371 as the Apps proposal. The card revision is read through; Bundle revision rules are unchanged. - The change is a connection test.
min_improvement_pctis only an input on that inconclusiveevaluatecall.
Not verified here:
- Tests were not executed, and CI was not inspected.
- Bodies of
KnowledgeReader.read,execute_trial, andevaluatewere not re-opened beyond these call sites.
Optional follow-up (non-blocking): OfflineBenchmarkAssessment.assess installs a unittest.mock tripwire on subprocess.run and Popen around evaluate. That guards this assembler. A local from subprocess import run binding inside evaluate would slip past that patch.
amy-why-3459
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed base f6d2007 and head a0a2d81 (new PR, full diff).
Verified:
- The
tests/architecture_real_ports.pycalls match the current signatures at this head:KnowledgeReader.read(reference),execute_trial(command, cwd, trial_dir, deadline, metric_key)withmetric_key=None(so no metric is read and onlyexecution.jsonis written), andevaluate(spec, baseline, candidate, validation, integrity). - The
INCONCLUSIVEoutcome follows fromharness/evaluation.pyitself. It gets an expected/recorded trial-count mismatch (candidate is empty), a missing metric,validation=None("quality validation command was not supplied"), andintegrity.verified=False(the supplied snapshot reason).improvement_pctstays None, so the single CPU record cannot be read as a PASS. tests/architecture_closed_loop.pyonly has docstring changes, and the coordinator still imports noinferrsi*modules. The new AST test pins the ports to the three existing seams and rejectscli/arguments/harness.runner/apps/knowledge/benchmark,collect, andsearch_knowledge.docs/module-layout.mdis consistent with the code.MemoryArchiveholds only prepared and assessment bytes, and the record digest is not inarchive.prepared.- All 7 check runs on this head passed: declarations, pr-references, full-check 3.10/3.11, installed-package 3.10/3.11, lint-and-smoke.
Not verified: I did not run the tests locally. I relied on the CI results above.
Optional (non-blocking): "exit_code": record["returncode"] in the evidence duplicates returncode. If nothing downstream consumes it, it could be dropped to avoid confusion with the envelope exit-code table, which is still undecided.
zengchuang-hw
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed base f6d2007 and head a0a2d81.
Delegation: zengchuang-hw directly authorized this OpenAI assistant to act as an independent reviewer for PR #378, reply with the review result, and submit Approve on PASS. This review is submitted under zengchuang-hw; it does not authorize integration or Issue acceptance.
No blocking findings in this bounded test/documentation slice.
- Traced the real KnowledgeReader.read, execute_trial, and evaluate paths. The coordinator remains port-injected; the retained record binds the prepared digest, and the assessment retains the knowledge revision.
- The local CPU process return code remains an execution fact. Missing paired measurements, validation, and source integrity correctly preserve INCONCLUSIVE.
- Local exact-blob snapshot: PYTHONPATH=src python3 -m unittest tests.test_architecture_closed_loop tests.test_target_module_import_gate passed all 19 tests on Python 3.12.14.
- Rechecked all 7 CI jobs and knowledge-model-policy: successful. Ready CI full-check on Python 3.10/3.11 passed; logs show 3560 passed, 14 skipped and 286 subtests passed per version, with Ruff passing. The tested merge tree matches this head.
Limits: the local snapshot was focused; full pytest and Ruff were verified through CI, not rerun locally. MemoryArchive and refusal-path injected outcomes remain test substitutes. No production Apps assembly, persistence/restart replay, full module isolation, hardware/model execution, or campaign acceptance was validated. Issues 372 and 362 remain open.
CI: https://github.com/InferMatrixLab/InferRSI/actions/runs/42743431583
Governing Issue and delivery slice
Refs #372
Refs #362
5999ced639a88d359dfd7bb1798fef9548717eb5.f6d200775d554d08fb73673833611dd6ac02d14b. Those ports were injected fakes.tests/architecture_real_ports.pyis injected into that same handoff for the success connection. It callsKnowledgeReader.readon the installed package cardtransformers.common.llm.kv_cache_strategy_contracts(record.json only),execute_trialfor one local Python process with no device, andevaluateon the retained record. The prepared bytes carry the knowledge revision, the harness fact carries the prepared digest, and the assessment bytes carry the same revision plus the process return code. The coordinator intests/architecture_closed_loop.pystill does not import the three modules.knowledge/,benchmark/, orapps/tree, no Bundle revision change, nocollectorsearch_knowledgechoice, no archive-owner or RunStore choice, no new exit-code table, no CLI or runner edit, no layout migration, and no corpus or fixture move. Refusal cases still inject outcomes the single CPU record does not produce. Issues 372 and 362 remain open.Knowledge authorship declaration
Not applicable. This PR does not change knowledge content or Bundle revision rules. The test-local JSON digest is not
inferrsi-json-v1. The read uses the existing package card and returns its current revision; that revision is not pinned in the tree.Changed artifacts and evidence
PackageKnowledgeRead.delivercallsKnowledgeReader.readtransformers.common.llm.kv_cache_strategy_contracts. A second read returned the samesha256:revision. Bundle content wasrecord.jsononly. Passedsearch_knowledgestays unsetCpuOfflineHarnessRecord.executecallsexecute_trialsys.executableprocess wrotetrial/result.json(probe=cpu-offline,device=null) andtrial/execution.json(status=completed,returncode=0, pid present,elapsed_seconds> 0). No device requested. Passedcpu-offline-1is allocated by the test port.runner.pyis untouched because #367 is openOfflineBenchmarkAssessment.assesscallsevaluateINCONCLUSIVE.improvement_pctis null.statistical_significanceisnot established. Reasons include the missing validation command andcpu offline record has no source snapshot. Assess patchedsubprocess.runandPopenand did not start a process. Passedrun_looppreparedstores the knowledge revision. The record fact stores that prepared digest. The assessment stores the same revision andrecord_returncode=0. The record digest is the fact hash, and it is absent from the prepared archive. PassedMemoryArchive. RunStore ownership stays unsetPYTHONPATH=src python3 -m unittest tests.test_architecture_closed_loop tests.test_target_module_import_gate: 19 passed in 0.397sknowledge_operations,harness.scenarios.external, andevaluation. They do not importcli,arguments,harness.runner, or the absentknowledge,benchmark, orappspackages.knowledge_operations.pyandexternal.pydo not import each other. Passedevaluation.pystill re-exportsharness.evaluation. That existing edge is unchangedpython3 -m ruff checkon the three Python files passed (ruff 0.17.4).git diff --checkcleanruff check src tests toolsand full pytest were not runChecks and dependencies
PYTHONPATH=src python3 -m unittest tests.test_architecture_closed_loop tests.test_target_module_import_gateon CPU, with no device: 19 passed (15 closed-loop, including the real connection, and 4 import-gate) in 0.397s. Ruff 0.17.4 passed ontests/architecture_closed_loop.py,tests/architecture_real_ports.py, andtests/test_architecture_closed_loop.py.git diff --checkwas clean.ruff check src tests tools, Ready CI, hardware, GPU/NPU, and model runs were not run. No workflow file was edited.cli.py,arguments.py,harness/runner.py, andpyproject.toml. This branch does not edit them. task: Apps 应用层详细设计与迁移计划(CLI / Python / Plugin) #371 still names the Apps package. The assembler stays undertests/.Remaining work and Issue closure
search_knowledge, archive ownership versus RunStore, and the Apps directory name.Handoff (only when an actual handoff or decision changed)
a0a2d81082d44e0fb8e1b7561d2a39a7db657398.