Repository navigation
knowledge: audit Infera serving-search result provenance - #332
Conversation
Add an implementation-bound reference for result provenance, deterministic seed comparison, complete evaluated-trial history and future trace-bound ranking calibration in the existing knowledge owners. Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com> Agent-assisted-by: Codex / gpt-6.1-sol [effort=max] (author)
Retain valid duplicate history rows in the Infera source explanation and separate signature-based future evaluation skips from loading the ledger. Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com> Agent-assisted-by: Codex / gpt-6.1-sol [effort=max] (author)
Build the native catalog and coverage from the corrected canonical source revision while preserving all existing owners and source records. Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com> Agent-assisted-by: Codex / gpt-6.1-sol [effort=max] (author)
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Skill evaluationPLANNED: not executed; there are no task outcomes yet.
Author verification steps (nonpaid; not yet run)
python tools/build_skill_index.py --source-revision 8d40de5673666430573a5d1c1aa3f17e906a1be3 --checkThe declared source pin matches this head's canonical library bytes. The command has not run: retain its exit code and full drift/success output. It checks the generated projection, not claim truth or skill efficacy.
python -m json.tool optimization-search-policy/plan/tasks.json
python -m json.tool optimization-search-policy/plan/schema.jsonUncovered scope by location: Canonical library paths: 4; Generated projection paths: 81; Other paths: 3. Path counts are not claim counts; no metadata-only conclusion is inferred. Evidence and logs · Retention: 30 days. Plan bindings, budget, frozen exam and advisory detailsAdvisory: obtain missing evidenceRetained evidence availability: planned / unexecuted. No approval or merge authorization is issued.
Model/settings:
Frozen exam hashes: Selection/task metadata (not a relevance verdict): {"adapter":"kv-admission-diagnostic-v1","basis":"trusted_path_mapping","matched_paths":["src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md","src/inferrsi/data/skill_library/methods/optimization-search-policy/metadata.json"],"required_answer_fields":["diagnosis","evidence","next_check","reuse_authorized"],"tasks":[{"id":"case-05","provenance":"Synthetic finite-state observations; not a production trace or engine benchmark.","request":"Diagnose this blocked request at the recorded decision boundary; propose the next read-only check.","split":"holdout"}]}Automatic execution / cumulative PR budget: {"reason":"Trusted configuration or pricing missing, invalid or expired","remaining_usd":"unknown","reserved_or_unresolved_usd":"unknown","spent_or_legacy_usd":"unknown","state":"blocked"}Projection check selection (not an executed result): {"declared_in":"src/inferrsi/data/repo_packs/inferrsi/knowledge/README.md","source_revision":"8d40de5673666430573a5d1c1aa3f17e906a1be3","status":"ready"}Provider/settings: Uncovered / unsupported paths (88)
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 16a296585e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| | Original source / version | Inspected scope and local evidence | Mechanism / consumer | Evidence limits | | ||
| |---|---|---|---| | ||
| | Vidur — [2405.05465v2](https://arxiv.org/abs/2405.05465v2) | [A readset](../src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/READSET.md); §§3–7, A.1–A.2 | mined — operator/queue model calibration and joint-SLO shortlist → [calibrate-serving-simulation](../src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/SKILL.md), ceiling analysis | Empirical prediction, not physical bound; near-capacity CPU overhead and 12.65% maximum error in the specified appendix case; no current runtime calibration | | ||
| | [AMD Infera @ ff1136e4](https://github.com/AMD-AGI/Infera/tree/ff1136e4a51998eba2933ed63cfb9c10120f86ea) | Direct bounded source inspection, 2026-11-03: serving CLI/evaluator/history, seed/objective parser, launcher/DES countercheck and named unexecuted test definitions; [16-path readset and rights](../src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/references/infera-serving-search/source-record.json) | mined — one [calibration-owned provenance reference](../src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/references/infera-serving-search/README.md); search-policy links its seed comparator and evaluated-trial ledger | Source-derived candidate; empirical not_run. Scalar tuner scoring and DES percentiles are separate; missing SLO fields and stale no-P/D-queues text are qualified. Selected original paths MIT; no runtime, ranking accuracy, quality, service or agent-benefit result. | |
There was a problem hiding this comment.
When generating the coverage view, tools/build_skill_coverage.py::references() recognizes owners only through backticked registered IDs or links to /SKILL.md. This row has neither: its local links target resource files, so the builder associates Infera with no skill. Consequently, the regenerated docs/skill-coverage.md still lists Vidur as the calibration skill's only mapped source, omitting the new Infera provenance. Include a backticked calibrate-serving-simulation mention or a link to its skill entry and regenerate the coverage view.
AGENTS.md reference: AGENTS.md:L100-L100
Useful? React with 👍 / 👎.
Name the existing registered calibration owner in the reference matrix so native coverage retains the Infera source mapping. Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com> Agent-assisted-by: Codex / gpt-6.1-sol [effort=max] (author)
Dong1017
added
the
area:knowledge
ChatGPT Codex Connector
Generate native projections from actual corrected canonical source f8a7e83. Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com> Agent-assisted-by: Codex / gpt-6.1-sol [effort=max] (author)
|
This docs-only PR adds an owner-local Infera serving-search provenance reference under Reviewed commit: 0a215de |
agent-ci-infra-bot
Bot
added
bot-reviewed
kind:docs
|
Review at head
Not verified by me: the upstream source pins in |
|
@amy-why-3459 @zengchuang-hw, could you please independently review this PR at To address the cross-PR integration concern, I propose merging this PR first, followed by #322, #324, #325, #339 and #342, subject to each PR's current reviews and checks. After each predecessor lands, we will preserve the combined canonical sources, regenerate the shared index and coverage from the actual source revision, and revalidate the next head before requesting its review. Please also let us know if you suggest a different order. Source and software checks do not establish model, device, or effectiveness results. |
|
Thanks for the review request and proposed order. I did a Codex-assisted, non-voting integration check of
The current head is awaiting formal independent approvals. |
|
Update to my earlier integration check, for the original review/order request. Head remains A read-only trial merge now reports 81 conflicting generated files: Before integration
python tools/build_skill_index.py --source-revision <combined_source_sha>
python tools/build_skill_coverage.py
python tools/build_skill_index.py --source-revision <combined_source_sha> --check
python tools/build_skill_coverage.py --checkThe source pin must contain the combined canonical bytes. Regeneration should retain all 80 registered cards, including main's new
This is a non-voting integration follow-up by Codex through |
Preserve the current main tree and the eight Infera canonical changes. Regenerate the catalog and coverage in a separate commit from this source snapshot. Agent-assisted-by: Codex / gpt-6.1-sol [effort=max] (author) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
|
Coverage review at head
Not verified by me: upstream source pins in Reviewed commit range: |
Carry forward the synthetic CPU KV retirement and preserve the combined Infera sources. Keep the source snapshot available for native catalog and coverage regeneration. Agent-assisted-by: Codex / gpt-6.1-sol [effort=max] (author) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
Bring in the DSH plugin changes while preserving the combined Infera source. Agent-assisted-by: Codex / gpt-6.1-sol [effort=max] (author) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
Run the current main writers on the committed combined source snapshot. Retain the full registered catalog and source mappings. Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=xhigh] (author) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
PR review verdict — DO NOT PASS
Reviewed base 3a67f2456156f1071d4821e172a5bb55393f3f08 and head 5656e04e86f8d0f4a6741984436868991de77877 (full diff; this seed has no previous head).
Prior finding
- Resolved: Codex P2 on
16a296585e(docs/reference-matrix.md, Infera row not associated with a registered skill). This head backtickscalibrate-serving-simulationin that row, and regenerateddocs/skill-coverage.mdlists both Vidur and AMD Infera @ ff1136e4 for that skill (candidate · not_run · 3 readset(s)). Input digest issha256:a42c2cdd0c173f0c50258e4344ff07ad055d40413c520757b16b16caa479ea51.
Blocker
src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/optimization-search-policy.jsonrecords new source digests: SKILL.mdd03957ba8f5ff3892947a5c3a45f46de21145bdad3bfb1cbed26faeb172de7a9(was0ba4ade1648764a7422f564abe2d4fc9f9a25b5943a6c70c81ce17b8e7fb48d9) and metadata.jsonc12db77c3536dc3b08635507efcf97c577dffc343186a953d32e5314380896df(was6dc0c3b8ce13cff285ad99da4441f2fb6a5931247e7634e657275dd915854e02). The same record appends links to../calibrate-serving-simulation/references/infera-serving-search/README.mdandsource-record.json. The full base..head diff does not modifysrc/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.mdormetadata.json(the patch ends insidecalibrate-serving-simulation/references/infera-serving-search/source-record.json). Those digests do not identify the optimization-search-policy bytes in this head.docs/reference-matrix.mdanddocs/searchspace-matrix.mdstill say search-policy links the Infera seed-only comparator and evaluated-trial ledger.- Expected correction: commit the skill and metadata edits that produce those digests, or regenerate the pack from the tree that is actually in this PR, then re-check
python tools/build_skill_index.py --source-revision 8d40de5673666430573a5d1c1aa3f17e906a1be3 --check. The pinned snapshot, the committed sources, and the recorded digests need to be the same tree.
Verified
- Infera provenance is owner-local under
calibrate-serving-simulation.SKILL.mdsends serving-tuner shortlists throughreferences/infera-serving-search/README.md. Skill metadata and the packaged record add theff1136e4a51998eba2933ed63cfb9c10120f86eareadset (16 paths) and provenance readsetssource-notes.md,infera-serving-search/README.md, andsource-record.json. - Visible claim locators I1–I6 sit inside the recorded inspected ranges: cli.py 122–310, evaluator.py 525–718 / 850–1086 / 1360–1385, inference_tuning.py 724–871 / 1394–1519, inference_agent.py 328–366, launcher.py 258–322 / 463–515 / 989–1050 / 1178–1187, des.py 2329–2394 / 2638–2668 / 3070–3145. Full-text reads cover history.py, the cited manuals, and
test_tuning_slo.py. - Wording stays a source-derived candidate with empirical validation
not_run. Scalar tuner scoring and DES percentiles are separated; missing SLO fields and the stale no-independent-P/D-queues note are qualified. The ranking plan is proposed.
Not verified
- Did not run
build_skill_index.py --checkorbuild_skill_coverage.py --check, and did not recompute packaged content hashes. - Did not fetch AMD-AGI/Infera
ff1136e4blobs to confirm the recorded git blob SHA-1 or SHA-256 values. - Did not confirm snapshot
8d40de5673666430573a5d1c1aa3f17e906a1be3is contained in head5656e04e. - Patch lines 1912–1913 (
source-record.jsonrights.selected_path_coveragetail) were unreadable in this seed. The 16 readset keys earlier in that file were counted.
Optional
- The history/ledger paragraph in
references/infera-serving-search/README.mdis one long line. Wrap is cosmetic.
This comment does not approve and does not change scope. I will re-read after the author pushes.
zengchuang-hw
left a comment
zengchuang-hw
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Incremental range 0a215de89a7cd9023d022fd5e9f4d943ef5ebf99 → 5656e04e86f8d0f4a6741984436868991de77877 on main at 3a67f2456156f1071d4821e172a5bb55393f3f08. The new head clears the previous integration review.
Resolved in this review（本轮已解决）
- The new head descends from both the last reviewed head and current
main. My trial merge produced the head's treef536a6dc21a54a07b719077480e1a91855c65e5ewithout conflicts. The eight Infera source edits remain present alongside main'svllm-ascendregistration. Regeneration is confined to the 80 skill records, generated knowledge README and coverage view. The coverage view still maps both Vidur and Infera tocalibrate-serving-simulation. Integration follow-up · Coverage review - The complete tree comparison includes
optimization-search-policy/SKILL.mdandmetadata.json. Their SHA-256 digests match the generated record at both5656e04eand the committed source snapshot8d40de56. The index check reproduced the current records, which closes the digest discrepancy reported from the truncated patch. Cursor review · Generated record
Before I can approve（批准前必做）
No outstanding items for this revision.
Non-blocking notes（非阻塞问题或建议）
The retained history, seed comparator, missing-SLO behavior and scalar/DES distinction are consistent with the pinned sources. The independent trace-ranking study remains proposed under the source-only scope. Issue #91 · Research record
Verification and limits
git diff --check, the current-main trial merge,build_skill_index.py --source-revision 8d40de5673666430573a5d1c1aa3f17e906a1be3 --check(80 records), andbuild_skill_coverage.py --checkpassed. I independently verified the upstream commit tree and all 16 readset files by Git blob SHA-1, byte length and SHA-256. Pinned source record- Focused local pytest suites for the knowledge library, catalog, index, coverage and CLI passed in an isolated checkout of this head.
- Ready CI passed on the merge tree identical to this head, including full and installed-package checks on Python 3.10 and 3.11 and PR references. The current
knowledge-model-policycheck passed. SkillEval produced a plan and execution was skipped. Upstream tests, model/device execution, empirical calibration, service performance and agent efficacy remain unverified. Ready CI · Policy · SkillEval plan
Direct delegation: zengchuang-hw authorized Codex as the acting independent reviewer through the connected zengchuang-hw account. The delegation covers inspecting this PR, posting review findings, and submitting a formal Approve for the reviewed commit when the content and applicable checks pass. Original review request
amy-why-3459
left a comment
amy-why-3459
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed base 3a67f2456156f1071d4821e172a5bb55393f3f08 (current main) and head 5656e04e86f8d0f4a6741984436868991de77877. Incremental range since my last review: 0a215de89a7cd9023d022fd5e9f4d943ef5ebf99 → 5656e04e86f8d0f4a6741984436868991de77877.
Earlier findings
- Resolved: current-main integration blocker from my coverage review. The new head descends from both
0a215de8and current main3a67f245(merge commitsc5540aae,4f1ec3dd,8d40de56, then regeneration5656e04e; no history rewrite). GitHub reportsmergeable=true. Relative to main the PR now touches exactly the 8 canonical paths plus 82 generated paths (80 skill records, generated knowledgeREADME.md,docs/skill-coverage.md). - Not a defect: the digest blocker in the cursor[bot] review.
optimization-search-policy/SKILL.mdandmetadata.jsonare in the PR file list (+2/-0 and +3/-1). Their SHA-256 at5656e04eand at8d40de56ared03957ba…andc12db77c…, matching the generatedoptimization-search-policy.json; main's bytes are the old0ba4ade1…/6dc0c3b8…. The report came from a truncated patch.
Verified
- All 80 generated skill records name source commit
8d40de5673666430573a5d1c1aa3f17e906a1be3. - Every
source_filesdigest in the generatedcalibrate-serving-simulation.json(6 files, including the new Infera README andsource-record.json) matches the head bytes. - Hosted checks on this head passed: full-check and installed-package on Python 3.10 and 3.11, pr-references, plan/publish. The latest
declarationsrun passed (an earlier run at 01:50 UTC failed before the body update and was superseded at 01:54 UTC). - Content of the Infera reference was already reviewed without content defects at
0a215de8; the canonical Infera bytes are carried unchanged through the merges per the PR description and the matching digests above.
Not verified by me
- I did not run
build_skill_index.py --check/build_skill_coverage.py --checklocally on this head; I rely on the hosted checks and the digest spot checks above. - Upstream source pins in
references/infera-serving-search/source-record.json(zengchuang-hw reports verifying them independently).
Non-blocking: later knowledge PRs (#322/#324/#325/#339/#342) will need to regenerate projections from the combined source after this lands, as already noted.
Refs #91. Refs #177.
Problem and scope
An Infera result labeled
source="inference"can represent dry, analytical or anchor-assisted paths. Its best-summary output, retained history and scalar SLO checks also describe different evidence. This source-only amendment adds one implementation-bound reference to the existingcalibrate-serving-simulationowner and a short route fromoptimization-search-policy.The reference distinguishes measured anchors from target projections, a deterministic seed sweep from an LLM continuation, retained evaluated rows from future duplicate-evaluation skips, and the scalar parser from the separate DES percentile report. It preserves missing/unknown SLO-field limits and the code/documentation disagreement about split P/D queues. Trace-bound ranking calibration is an explicitly proposed independent study, with no supplied runner, calibration result or service guarantee.
Eight canonical paths contain two owner-local references, the two existing owner route/metadata pairs and the two matrix mappings. Existing Vidur knowledge and generic search rules remain intact. No registered skill, engine, planner, runtime, schema or experiment implementation is added.
Fixed source and provenance
ff1136e4a51998eba2933ed63cfb9c10120f86ea, tree7ca812406c61b52452270c4c8c53d1582d510376.d4a1922a452ed5b10f4cbd303d4f1d90e871f4b9and corrected M2fe3891c8db8c0fe38c8d5fd97b287ba19de82cf9over main663f3b327998526a13aa31e72ad729025c18983c. Canonical M3f8a7e832feab05f8262ebe4057f34e2c8a48fd95adds the existing calibration owner ID to the reference-matrix row so the original coverage parser attributes Infera correctly.Current-main integration (head
5656e04e86f8d0f4a6741984436868991de77877)This addresses the integration follow-up and the coverage review. The branch fast-forwards from the previous head
0a215de89a7cd9023d022fd5e9f4d943ef5ebf99; no history was rewritten.c5540aae7eaa964fbde8fd1047a39b1d096d44045c86eb57vllm-ascendowner and registry entry4f1ec3dd405f539e445b173f348d7691d97042aac5540aae, main1246348f8d40de5673666430573a5d1c1aa3f17e906a1be34f1ec3dd, main3a67f2456156f1071d4821e172a5bb55393f3f084f1ec3ddblobs and modes5656e04e86f8d0f4a6741984436868991de778778d40de56Relative to current main
3a67f245, the head changes exactly the eight canonical paths plus 82 generated paths: 80 registered owner records, the generated knowledgeREADME.mdanddocs/skill-coverage.md. Every owner record names8d40de56as its source commit.Commands run at this head, all exit 0:
git merge-tree --write-tree 3a67f245 5656e04eis conflict-free and produces exactly this head's treef536a6dc21a54a07b719077480e1a91855c65e5e. The coverage row attributes both Vidur and Infera tocalibrate-serving-simulation, andvllm-ascendremains registered and covered.Focused local tests at this head (Python 3.13, imports bound to the checkout) passed with exit 0: 162 passed, 5 skipped across the index, coverage view, catalog, library/readers, knowledge contract, catalog CLI, Collect selection and knowledge CLI suites. The worktree stayed clean. Hosted Python 3.10/3.11 full and installed-package checks are the full-suite evidence for this head.
Independent private gpt-6.1-sol / ultra static reviews passed the source merges
c5540aaeand4f1ec3dd(review-only). The final step to8d40de56was verified by script against the exact upstream delta and the unchanged canonical blobs; it was not separately reviewed. The regenerated head has not had an independent review yet; formal review is requested on this head.Earlier head evidence (
0a215de89a7cd9023d022fd5e9f4d943ef5ebf99)The following results apply to the previous head over main
663f3b32and are not substituted for current-head checks.Independent private gpt-6.1-sol / ultra source review passed corrected M2, including retained duplicate history rows versus future evaluation skips, and separately passed the minimal M3 owner-mapping correction against the original parser contract. Both original native write modes passed at M3, and the index/coverage checks returned zero at that head.
Native owner/catalog and resource readers passed for both affected owners. Checkout bundles contained 5 calibration resources and 26 search resources; each source-package bundle contained 757 resources. Fourteen revision-bound resource API reads covered the two new references, owner/package entry points and cross-owner routes. Whole-resource Collect returned both requested owners for both adapters with complete manifests.
Ready 41626479226 passed all six jobs. Python 3.10 and 3.11 full checks each reported 3,544 passed / 13 skipped / 1 warning / 284 subtests, plus Ruff. Both fresh isolated noneditable wheels loaded 127 source-grounded records. Independent private gpt-6.1-sol / ultra review of that complete head returned LGTM with no findings. Codex review completed without findings; the previous owner-mapping thread remains outdated and its source issue is corrected.
Limits
Knowledge remains candidate/not_run and efficacy remains not_evaluated. Source test definitions were inspected, not run. No upstream import/build/test/CLI, model/device/service execution, ranking calibration, agent comparison or paid trial was performed. No SkillEval comparison or model call was executed. Pending Xe-Forge, DeepSeek, NInfer, Faster CosyVoice and Embodied increments are not imported; each later PR must preserve its canonical increment on the actual predecessor and regenerate projections from the combined source revision.
Knowledge authorship declaration