[Dong1017]

Governing Issue and delivery slice

Refs #253. Follow-up deployment for merged #267; no actual comparison result or Issue completion is claimed.

• Manual main-only job, no dispatch inputs. The reviewed issue-scoped authorization artifact owns historical revisions, experiment ID, limits, selected skill and qualification identity. Main pins only its locator, author and complete body hash; shared workflow/controller contain no campaign revisions. Accounting remains separate, with a generic maximum of two requests and USD 20.34. The accounting PR is not represented as either historical skill revision.
• Trusted preparation freezes public task/starter/skill inputs, protocol and approval before existing provider-secret injection. Exact checkout/main/run/operator/first-attempt identities are checked; arbitrary PR code, configuration, archives or dispatch-supplied authorization are not accepted.
• Existing repository provider key is confined to the trusted execution step/provider worker; the repository token is excluded from that worker. Generation and capture reporting never execute the parser. Independent replay remains a separate secret-free workflow.
• Durable PR/day accounting includes previous reservations; stable experiment identity and first-attempt checks forbid retries. Unknown cost, missing key, changed/expired config, failed host qualification or modified freeze stop execution.
• Packages fixed public inputs, inert parser bytes and bounded numeric/status receipts, not raw logs or arbitrary model metadata.

Operational configuration

• Existing key's equal-price default/claude-code/claude-code-pro group is owner-attested, not independently inferred from public pricing or represented as an invoice audit.
• Public pricing evidence: [Tracking] SkillEval evolution: local evidence, PR review and optional LLM judge #253 (comment)
• Dedicated feat(skilleval): bounded cloud generation and isolated artifact replay #267 audited genesis: 9c53ce5f5f40b4765d64847e66c552d4857108e9; existing daily ledger ancestry is retained without reset or new reservation in this PR.
• Config expires 2026-10-30 16:00 UTC. Expiry never silently extends.
• Operational configuration review link points to this PR. Issue authorization is independently bound to comment 6518412099, owner author and complete body SHA256 ea6493151023f6167c833ad2099838c867d3d7548a612a7cc0b8769bb26fec4c; it is rechecked before reservation and every provider grant.

Checks and dependencies

• 27 new offline job tests PASS; combined job/accounting/generation selection 62 tests PASS; Ruff PASS; diff check PASS.
• Real historical Git-object preparation completed without a provider call. Fixed historical objects are fetched before secret injection; no historical code is checked out or run.
• Independent security review of the initial implementation found no blockers and independently passed 27 job tests.
• Initial tested tree: 1287fe75ca98179460c5e123f4b4a4e19374928d; final reviewed metadata-only tree: a6d24a924bb92b046f02c0edd431cfa8d2d1d54e; base f93df0548a26714f6514337a8525c7c14108acad (feat(skilleval): bounded cloud generation and isolated artifact replay #267).
• No paid calls or dispatch performed by these tests. Actual host qualification was separately established by https://github.com/InferMatrixLab/InferRSI/actions/runs/41017219756; independent artifact replay still requalifies its own host.

Knowledge authorship declaration

Agent-assisted code/workflow/operational configuration only; no protected skill or knowledge asset edits. Model identifiers are not asserted.

Remaining work

After exact-head review and passing normal CI, merge the bounded job, freeze final main inputs, perform durable admission and the two real captures, then independently replay and report the measured outcome. A negative or inconclusive result remains valid evidence; neither this software PR nor green CI closes #253.

Native-review correction pending final verification

P1 discussion_r4421672934 is addressed in proposed head 9fec28e4de261537757bdb4747566f7473a384c8 (tree bf2fc3880768c6c453e862b31c5fc658f8a400a8): campaign pins/defaults moved to the acceptance Issue; strict fixed-endpoint/author/issue/full-body-hash/duplicate-key validation and argv-only Git fetch preserve the trust boundary. The old in-tree campaign policy is deleted. New synthetic-fixture and revocation-before-reservation tests are added. After the separate CPU measurement completed, the corrective delta passed 70 local tests (job/accounting/generation) and Ruff. A real public-comment preflight fetched and verified the pinned issue record, fetched only validated historical Git objects, and prepared the frozen plan with the provider key removed; no model call or ledger write occurred. Current-head remote CI and review remain required before merge; no substantive review thread has been resolved by this update.

[github-actions]

Skill evaluation

Automatic execution / cumulative PR budget: {"reason":"Trusted configuration or pricing missing, invalid or expired","remaining_usd":"unknown","reserved_or_unresolved_usd":"unknown","spent_or_legacy_usd":"unknown","state":"blocked"}

Advisory: obtain missing evidence

Retained evidence availability: planned / unexecuted. No approval or merge authorization is issued.

• Retained evidence is planned / unexecuted.
• No supported comparison cases were selected.
• Uncovered/unsupported scope needs a case or explicit reviewer disposition.

PLANNED: execution is not authorized by knowing this hash.
Outcomes: {"fix":0,"inconclusive":0,"missing_pair":0,"regression":0,"unchanged_correct":0,"unchanged_incorrect":0}
Earlier revision results are stale for this plan.

Selected comparisons: 0 · Estimated task calls: 0.
Uncovered/unsupported changes: 9. Gaps never count as passes.

Applicable only while these revisions and inputs match the current PR.
Paid execution requires trusted label authorization and cumulative PR reservation. Unknown accounting blocks new paid calls; planning is not execution.
Pipeline evidence only; offline fixed outputs do not establish skill efficacy.
Evidence and logs · Retention: 30 days.
Outcomes are historical; verifiable only before Unix time 1795940827 while the linked artifact is available. Expired evidence is unverified.

Plan bindings, budget, frozen exam and advisory details

Unchanged-correct is valid neutral evidence, not incremental benefit.
Content/source review and authorship admission remain separate.
Trusted regrading checks retained answers, not model/hardware provenance or a hostile executor; this is not an efficacy certification.

Per-case outcomes (discovery excluded; regressions and gaps first):

[]

Showing at most 20 of 0 pairs; the evidence link contains all outcomes, including omitted regressions/gaps.

Plan SHA-256: cefeaf534edfd51c45639350d21e8d9ed993829f8f8d5edcf17e6b352ebcb58d
Base: f93df0548a26714f6514337a8525c7c14108acad · Head: 9fec28e4de261537757bdb4747566f7473a384c8
Trusted implementation: f93df0548a26714f6514337a8525c7c14108acad

Model/settings: {"effort":"not_applicable","name":"claude-opus-5-5"}
Budget: {"comparisons":0,"estimated_calls":0,"limits_per_comparison":{"campaign_seconds":600,"max_task_calls":2,"repeats":1,"timeout_seconds":180,"token_stop_new_trials":200000},"provider_config_sha256":"b28d80f8194fbe0f5a9f01515efb24b218979fd45f702776560da276f4b30d04","reservation":null}

Skill	Comparison	Cases	Estimated calls

Provider/settings: {"client":{"api_version":"2023-06-01","endpoint":"https://yibuapi.com/v1/messages","kind":"anthropic-messages-full-context-v1","max_output_tokens":4096,"max_read_bytes":32768,"max_request_bytes":262144,"max_requests":1,"max_tool_calls":12,"provider":"yibuapi"},"limits":{"token_stop_new_trials":200000},"model":{"effort":"not_applicable","name":"claude-opus-5-5"},"pricing":{"currency":"USD","expires_at":1793271228,"group":"default","max_input_tokens":1000000,"model":"claude-opus-5-5","rate_evidence":"https://github.com/InferMatrixLab/InferRSI/issues/215#issuecomment-6467531341","rates":{"cache_read":"0.4","cache_write":"10","input":"8","output":"40"},"request_fee":"0","unit":"per_million_tokens"},"spend_limit":{"amount":"200","currency":"USD"}}

Uncovered / unsupported paths (9)

• uncovered: ".github/workflows/skill-eval-historical.yml"
• uncovered: "docs/operations.md"
• uncovered: "docs/skill-eval-historical-job.md"
• uncovered: "tests/test_skill_eval_historical_accounting.py"
• uncovered: "tests/test_skill_eval_historical_job.py"
• uncovered: "tools/skill_eval_historical_accounting.py"
• uncovered: "tools/skill_eval_historical_authorization.json"
• uncovered: "tools/skill_eval_historical_config.json"
• uncovered: "tools/skill_eval_historical_job.py"

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-30T08:28:08.257221Z	9fec28e	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 961731057e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Keep historical campaign revisions out of shared automation

The workflow commits the actual baseline and candidate campaign SHAs directly into its fetch command, with the same campaign pins duplicated in the new controller and policy. The root AGENTS.md explicitly requires campaign revisions to remain issue-scoped and forbids committing real campaign SHAs into workflows. Move these revisions into an issue-scoped, reviewed authorization artifact and have the trusted preparation path validate that artifact, preserving the existing authorization boundary without embedding this experiment in shared automation.

AGENTS.md reference: AGENTS.md:L54-L58

Useful? React with 👍 / 👎.

[Dong1017]

Scoped software LGTM at 9fec28e / tree bf2fc3880768c6c453e862b31c5fc658f8a400a8, base f93df05.

P1 disposition for discussion_r4421672934: fixed and verified, not dismissed. Real historical revisions, experiment ID and numeric limits now reside in the issue-scoped record #253 (comment). Shared workflow/controller no longer embed those campaign pins; the former shared policy is removed. Main retains only the exact Issue/comment/author/full-body digest locator. Independent readback hashes the actual comment to ea6493151023f6167c833ad2099838c867d3d7548a612a7cc0b8769bb26fec4c. Fixed-API, strict schema/duplicate-key checks, no redirects, validated Git arguments, and revalidation before reservation plus every broker grant preserve the authorization boundary.

Verification: 70 corrective focused tests independently passed and Ruff passed; a no-key/no-model/no-ledger-write smoke fetched the actual Issue record and prepared the historical inputs successfully. Current-head Ready CI 41026464012 succeeded; native review 6518229123 completed on this head with no unresolved findings. Prior P1 is explicitly reconciled above. No other unresolved opinions identified.

This approves software readiness only, not efficacy or a completed trial. The manual main-only job still requires current Issue authorization, effective-rate validity, credential presence, successful qualification and durable budget admission. Generated code is never executed in the credential-bearing generation step, and the independent replay remains credential-free. No paid model call has run yet. Existing generic PR SkillEval advisory remains zero-comparison/unexecuted and is not used as evidence of benefit. #253 remains open. COMMENT, not a formal APPROVE.