Add Qwen3.8 FP8 correctness and service qualification workflows

docs/architecture.md

@@ -122,7 +122,10 @@ benchmarks/harness/         Compatibility command entrypoints
 | Hardware | `infra/platforms`, `data/platform_packs` | Device selection, process exclusivity and independent backend/model evidence |
 
 The acceptance rule is: **a new model or hardware adapter must reuse the common
-AB/BA loop, process cleanup and performance adjudication.** The extension
+AB/BA loop, process cleanup and objective-specific adjudication.** Performance
+remains the default; explicit correctness experiments require paired output
+artifacts and validation without metrics or gain claims. See the
+[correctness contract](operations.md#correctness-only-experiments). The extension
 regression test composes a new metric schema, non-Git source adapter and backend
 probe through those interfaces while running real subprocesses. Its numbers are
 synthetic contract fixtures, not accelerator measurements.

docs/operations.md

@@ -89,6 +89,30 @@ upstream instructions.
 - **Failure behavior:** ambiguous identity names the qualified choices; unavailable/stale revisions and missing/unsafe resources fail explicitly — no fallback to newer bytes.
 - **Example:** `inferrsi read calibrate-serving-simulation --skill-root src/inferrsi/data/skill_library --material`
 
+## Correctness-only experiments
+
+Use `inferrsi plan` or `inferrsi run` with `--objective correctness` for paired
+reference/candidate output validation. The default objective remains
+`performance`; its positive finite metric and gain gates are unchanged.
+`--repo-type nano_vllm` describes caller-supplied commands, not model support.
+
+Correctness trials retain alternating AB/BA order, source snapshots, deadlines
+and process cleanup. Each command must write a nonempty JSON object to `{result}`
+or `INFERRSI_RESULT_PATH`. No metric is extracted and no gain is computed.
+The explicit `--validation-command` receives `{run_dir}` substitution so it can
+read every fresh baseline/candidate trial and its output artifacts. It must
+verify the frozen case coverage, model/input identities and numerical gates;
+the generic runner only checks the presence of readable JSON objects.
+
+For correctness validators, exit 0 accepts the supplied checks, exit 1 rejects
+them, and exit 2 declares incomplete evidence. Other nonzero exits preserve the
+existing rejection behavior; validators must catch execution/dependency errors
+and return 2 rather than confuse a crash with a completed rejection. Timeouts,
+cancellation, missing outputs or validation, incomplete trial sets, and source
+drift remain `INCONCLUSIVE`. Missing evidence takes precedence over rejection.
+Correctness results cannot establish performance or service claims, and an
+individual run does not replace campaign-level completeness checks.
+
 ## Store candidates (Python)
 
 ### Create / Update — `KnowledgeStore.create` / `.update`

evidence/qwen38-h20-fp8-text-phase1/patch/README.md

@@ -0,0 +1,4 @@
+# Source identities
+
+Record `<control_sha>`, `<baseline_sha>` and `<candidate_sha>` in the acceptance
+issue and frozen runtime bundle. This preparation contains no revision pins.

evidence/qwen38-h20-fp8-text-phase1/plan/README.md

@@ -0,0 +1,49 @@
+# FP8 text engine and service preparation
+
+[Acceptance #57](https://github.com/InferMatrixLab/InferRSI/issues/57) owns this
+separate FP8 campaign under [parent #53](https://github.com/InferMatrixLab/InferRSI/issues/53).
+The historical BF16 preparation and evidence remain in their existing tree.
+
+Status: preparation only; INCONCLUSIVE. The workload is proposed, not frozen.
+Authenticate the official Qwen/Qwen3.8-27B-FP8 checkpoint at `<model_revision>`.
+Freeze its byte manifest, this workload, control/baseline/candidate revisions,
+registered environment lock, exact GPU, validator sources, gates and budget in
+the acceptance issue before any candidate experiment. No pins or numerical
+gates are supplied by this template.
+
+The checkpoint uses dynamic e4m3 FP8 linear weights with 128 by 128 blocks and
+BF16 activations; unquantized layers remain BF16. Reference and candidate must
+retain and execute FP8 weights. Whole-checkpoint BF16 dequantization is excluded.
+The reference additionally requires an immutable kernel-registry revision and
+trusted source fingerprint for the prepared torch-cuda build of
+kernels-community/finegrained-fp8, with offline caches and DeepGEMM disabled.
+
+The bounded `native_text_smoke` compares full-prefix Transformers teacher forcing
+with the native hybrid-state engine through the common correctness loop.
+
+The accepted phase-1 scope also includes the public engine entrypoint and a
+loopback-default HTTP text-completion service with non-streaming and SSE output.
+The service supports a documented greedy-generation subset. Unsupported fields
+and values must fail explicitly. Model calls may serialize while independent
+requests interleave; concurrency support is not a throughput claim.
+
+The separate `native_service_functional` qualification compares public native
+serial generation with HTTP/SSE serial and concurrent output. It must cover
+normal completion, EOS and token limits, explicit cancellation, client disconnect,
+malformed and over-limit requests, execution errors, shutdown and subsequent
+reuse. Cancellation is cooperative at model-call boundaries. Every terminal path
+must release request-owned KV, convolution and recurrent state and admission
+capacity without affecting another request.
+
+Record ordered output tokens and SSE events, request IDs, terminal reasons,
+overlapping admissions, state ownership, capacity and owned-process cleanup.
+Logical request-state bytes are distinct from allocator-reserved GPU memory.
+Freeze the service workload, protocol, source identities, admission limits and
+all readiness/request/release/shutdown deadlines before candidate execution.
+
+CPU fake-engine socket tests establish protocol coverage only. Final service
+qualification requires the real FP8 model and real HTTP sockets, separately from
+the reference numerical comparison. Missing required coverage is INCONCLUSIVE.
+Phase 1 includes L4 functional service evidence, with no performance gain,
+broad task-quality, continuous batching, stochastic sampling, vision, MTP or
+multi-GPU acceptance claim.

evidence/qwen38-h20-fp8-text-phase1/plan/workload.json

@@ -0,0 +1,58 @@
+{
+  "schema_version": 1,
+  "model_id": "Qwen/Qwen3.8-27B-FP8",
+  "tokenization": "raw_text_no_special_tokens",
+  "cases": [
+    {
+      "id": "english",
+      "prompt": "The purpose of a reproducible experiment is",
+      "continuation": " to make the methods and observations available so that another researcher can repeat the same procedure and compare the results.",
+      "max_steps": 8
+    },
+    {
+      "id": "chinese",
+      "prompt": "请解释为什么实验需要固定输入：",
+      "continuation": "固定输入可以控制变量，让不同实现的结果具有可比性，并帮助研究人员准确定位误差的来源。",
+      "max_steps": 8
+    },
+    {
+      "id": "code",
+      "prompt": "def add(a, b):\n",
+      "continuation": "    return a + b\n\nassert add(2, 3) == 5\n",
+      "max_steps": 8
+    },
+    {
+      "id": "arithmetic",
+      "prompt": "Compute 17 + 25. Explain: ",
+      "continuation": "17 plus 25 equals 42 because 10 plus 20 is 30 and 7 plus 5 is 12.",
+      "max_steps": 8
+    },
+    {
+      "id": "longer",
+      "prompt": "A laboratory records the same experiment on several days. Each record contains the input, model revision, runtime version, device type, numerical precision, output, and validation result. Some experiments change a kernel, some change batching, and some change memory layout. A reliable comparison controls every unrelated variable. When an experiment is interrupted, the missing results cannot be treated as successful. Failed trials are retained because they can explain which assumptions were wrong. The team separates correctness tests from service benchmarks and measures latency only after checking output quality. Explain how these practices help: ",
+      "continuation": "They make each comparison reproducible, preserve the evidence needed to identify regressions, and prevent incomplete experiments from being reported as successful optimizations.",
+      "max_steps": 8
+    },
+    {
+      "id": "english_replay",
+      "prompt": "The purpose of a reproducible experiment is",
+      "continuation": " to make the methods and observations available so that another researcher can repeat the same procedure and compare the results.",
+      "max_steps": 8,
+      "replay_of": "english"
+    }
+  ],
+  "precision": {
+    "weight_dtype": "float8_e4m3fn",
+    "activation_dtype": "bfloat16",
+    "quantization": {
+      "quant_method": "fp8",
+      "activation_scheme": "dynamic",
+      "fmt": "e4m3",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "dequantized": false
+  }
+}

evidence/qwen38-h20-fp8-text-phase1/results/README.md

@@ -0,0 +1,8 @@
+# Results
+
+INCONCLUSIVE: no FP8 campaign execution or qualification evidence is included.
+Retain every attempt and raw artifact after the acceptance issue freezes inputs.
+
+The accepted phase-1 scope includes public engine integration and HTTP/SSE
+functional lifecycle validation. Implementation checks and real-model service
+qualification must be reported separately; no service acceptance is recorded here.

evidence/qwen38-h20-fp8-text-phase1/validator/README.md

@@ -0,0 +1,17 @@
+# Validator preparation
+
+Freeze the example validator, profile helpers, trial adapters and offline kernel
+binding with their SHA-256 hashes before execution. Numerical gates belong to
+acceptance #57 and its frozen runtime records; this directory supplies no defaults.
+Missing model, precision, kernel or source evidence is INCONCLUSIVE. Complete
+numerical rejection is FAIL. Neither outcome may be relabelled as performance.
+
+Freeze the additional service validator, request fixtures and lifecycle protocol
+separately. Require exact deterministic serial/HTTP/SSE/concurrent token and text
+agreement, ordered terminal framing, actual overlapping request ownership,
+isolated cancellation and evidence of state/capacity release and subsequent reuse.
+An unobserved cancellation or overlap cannot pass because a request completed
+early. Missing observations, identity or cleanup evidence are INCONCLUSIVE;
+complete evidence of a protocol or isolation violation is FAIL. CPU fake-engine
+results cannot qualify the real FP8 model service. Service equivalence does not
+replace the independent reference-based numerical validation.

evidence/qwen38-h20-text-phase1/patch/README.md

@@ -0,0 +1,9 @@
+# Source identity
+
+Implementation references are recorded in acceptance Issue #54 before execution.
+The initial change adds an explicit correctness objective to InferRSI and a
+reference-self-check example. It does not adapt nano-vllm to Qwen3.8.
+
+Freeze the control revision, baseline/candidate revisions, validator hashes and
+dependency locks before measuring. Placeholder or dirty source identities cannot
+support a published model/engine acceptance result.

evidence/qwen38-h20-text-phase1/plan/README.md

@@ -0,0 +1,41 @@
+# Phase-1 acceptance preparation
+
+Parent: https://github.com/InferMatrixLab/InferRSI/issues/53
+Acceptance and numeric gates: https://github.com/InferMatrixLab/InferRSI/issues/54
+
+Status: **INCONCLUSIVE; NOT FROZEN**. No model or engine acceptance has occurred.
+
+The first executable milestone compares Transformers full-prefix and incremental
+cached execution on fixed raw-text teacher-forced tokens. It establishes a
+reference self-check only. Both sides use the same checkpoint and tokenizer.
+The independent nano-vllm-derived engine remains a separate incomplete milestone.
+Do not describe this bootstrap as engine support, broad model quality or a speedup.
+
+## Freeze checklist
+
+- [ ] BF16 download finished; all required files present.
+- [ ] Manifest anchored to immutable public model revision; actual weight and
+      config/tokenizer bytes verified against upstream identities.
+- [ ] Exact baseline/candidate source and control/validator revisions recorded in
+      the acceptance Issue, including dependency versions and imported paths.
+- [ ] Workload and validator hashes recorded; ensure each continuation produces
+      enough tokens for its requested steps without altering the frozen workload.
+- [ ] Numerical gates, resource budget and supported scope frozen in the Issue.
+- [ ] Actual single-GPU SKU, runtime and BF16 capability verified in allocation.
+
+`workload.json` is the proposed bounded smoke cohort: English, Chinese, code,
+arithmetic, longer prose and a repeated request to detect state contamination.
+It does not exercise chat templates, EOS stopping, stochastic sampling, batching,
+preemption, prefix reuse, MTP, vision, long-context limits or task accuracy.
+Those requirements must be covered before accepting the independent engine for
+its declared feature set. Unsupported paths must fail explicitly.
+
+The intended numerical gates from the Issue are per-case NRMSE <= 0.01 (reference
+RMS denominator floor 1e-8), top-1 agreement >= 0.99 and exact reset/replay equality.
+They are campaign parameters supplied to the validator, never library defaults.
+
+The initial allocation budget is one GPU, 8 CPU cores, 128 GiB host RAM and
+60 minutes; the paired experiment deadline is 45 minutes with two AB/BA
+repetitions. Model intake verification may need a separately budgeted CPU job.
+No optimization search has started. Do not relax gates after seeing candidates;
+a changed protocol requires a new freeze and fresh references.

evidence/qwen38-h20-text-phase1/plan/workload.json

@@ -0,0 +1,13 @@
+{
+  "schema_version": 1,
+  "model_id": "Qwen/Qwen3.8-27B",
+  "tokenization": "raw_text_no_special_tokens",
+  "cases": [
+    {"id": "english", "prompt": "The purpose of a reproducible experiment is", "continuation": " to make the methods and observations available so that another researcher can repeat the same procedure and compare the results.", "max_steps": 8},
+    {"id": "chinese", "prompt": "请解释为什么实验需要固定输入：", "continuation": "固定输入可以控制变量，让不同实现的结果具有可比性，并帮助研究人员准确定位误差的来源。", "max_steps": 8},
+    {"id": "code", "prompt": "def add(a, b):\n", "continuation": "    return a + b\n\nassert add(2, 3) == 5\n", "max_steps": 8},
+    {"id": "arithmetic", "prompt": "Compute 17 + 25. Explain: ", "continuation": "17 plus 25 equals 42 because 10 plus 20 is 30 and 7 plus 5 is 12.", "max_steps": 8},
+    {"id": "longer", "prompt": "A laboratory records the same experiment on several days. Each record contains the input, model revision, runtime version, device type, numerical precision, output, and validation result. Some experiments change a kernel, some change batching, and some change memory layout. A reliable comparison controls every unrelated variable. When an experiment is interrupted, the missing results cannot be treated as successful. Failed trials are retained because they can explain which assumptions were wrong. The team separates correctness tests from service benchmarks and measures latency only after checking output quality. Explain how these practices help: ", "continuation": "They make each comparison reproducible, preserve the evidence needed to identify regressions, and prevent incomplete experiments from being reported as successful optimizations.", "max_steps": 8},
+    {"id": "english_replay", "prompt": "The purpose of a reproducible experiment is", "continuation": " to make the methods and observations available so that another researcher can repeat the same procedure and compare the results.", "max_steps": 8, "replay_of": "english"}
+  ]
+}

evidence/qwen38-h20-text-phase1/results/README.md

@@ -0,0 +1,11 @@
+# Results
+
+Verdict: **INCONCLUSIVE**.
+
+No GPU experiment has run for this campaign. CPU contract tests, when recorded,
+qualify orchestration and rejection behavior only, not the model or accelerator.
+The BF16 checkpoint was not available during initial setup. Model readiness must
+be rechecked using the live downloader and byte-integrity preflight.
+
+Keep private raw runs outside tracked source. Publish only reviewed redacted
+copies, preserving hashes, failed attempts, scope and verification limitations.

evidence/qwen38-h20-text-phase1/validator/README.md

@@ -0,0 +1,12 @@
+# Validator freeze
+
+The proposed validator is `examples/qwen38_text/validate.py`, paired with
+`examples/qwen38_text/trial.py`. Before GPU execution, snapshot both files here
+(or in the private campaign bundle), record their SHA256 plus source revision in
+plan and acceptance Issue #54, and retain the complete model manifest/workload.
+This directory is currently a preparation record, not a frozen validator bundle.
+
+Validation checks complete paired trials, matching identities, full-vocabulary
+logit arrays, token coverage, finite values, file digests, numerical gates and
+reset/replay consistency. Missing evidence returns exit 2; a completed numeric
+rejection returns 1; scope-limited acceptance returns 0.