task: evolve a Qwen3.8-27B H20 engine with InferRSI; phase 1 FP8 text service #53

Current state: Active acceptance moved to #67, whose latest entry (2026-10-27) froze one v3 diagnostic with 900 GPU-seconds reserved and no posted outcome; InferRSI PR #55 is still an open draft at c69961b.
Decision needed: none
Next action: No assignee; the campaign continues in #67 within the owner-approved 44 GPU-hour ceiling, starting with the outcome of the reserved v3 diagnostic.

Active acceptance update — 2026-10-21

The owner authorized new acceptance #67 and administrative closure of #57 as superseded. #57's body, original numerical gates and historical FAIL/INCONCLUSIVE evidence are preserved; it was not accepted as completed.

Active acceptance: #67. Single-request streaming HTTP comparison against vLLM 0.32.0 with the same vLLM benchmark client, 8K/64K/128K inputs and 2K output. Both TTFT and TPOT must improve at every length. Task-quality acceptance uses tiered GSM8K/HumanEval+/MMLU evaluation; numerical metrics are diagnostic under the new contract only. Exact numeric acceptance gates and their pending fields are owned by #67.

Requirements are accepted, but runtime/data/validator identities and the full freeze are pending. First prepare bounded feasibility/cost estimates, then obtain owner confirmation of performance/retrieval gates and itemized GPU budgets before optimization search. No new model/performance PASS is claimed. This parent remains open.

The phase-1 scope and progress below are retained as historical records. They do not establish current PR merge state, active acceptance or completion.

Goal and acceptance

Build an independent nano-vllm-derived engine specialized for Qwen3.8-27B, using InferRSI for reproducible experiments and evidence-driven evolution. The accepted phase-1 slice establishes single-GPU FP8 text functionality and numerical correctness, public engine integration, and HTTP/SSE lifecycle and concurrent-request isolation, using a minimal InferRSI experiment loop.

Scope and constraints

• Phase 1: one allocated H20-family GPU, text input, official FP8 checkpoint execution, exact checkpoint/tokenizer identity, functional and numerical checks, public engine entry integration, HTTP/SSE generation, cancellation, state release and concurrent-request isolation. Verify the actual GPU SKU before qualification.
• Preserve Qwen3.8 hybrid DeltaNet/full-attention semantics, convolution/recurrent state, partial RoPE, chat template and sampling behavior.
• Establish trusted reference and candidate trial commands, immutable inputs, validator snapshots, source identities, AB/BA evidence and reports.
• Wait for model download completion and verify file bytes against a pinned manifest before model validation. No duplicate download or interference with unrelated allocations.
• Later slices: profiling, specialized kernels/scheduling, additional quantization formats, MTP, multi-GPU and vision; these are not phase-1 acceptance requirements.
• No performance improvement claim in phase 1. L0/L2/L3 evidence is required for the declared model-correctness scope; phase 1 additionally requires L4 functional service evidence. Future service-performance claims require separate frozen performance gates.
• Engine deployment remains independent of the InferRSI runtime. Reuse InferRSI's existing executor, cleanup and adjudication; avoid new CI lanes or GPU work in default CI.

Ownership and authorization

• Status: Accepted scope, explicitly authorized by the requesting repository owner in the development chat on 2026-10-21.
• Accountable parent: Codex in the originating chat; bounded implementation and independent review may be delegated to subagents.
• Goal acceptance and Issue closure: requesting owner; no automatic closure or merge authorization inferred.

Historical FP8 phase-1 acceptance and versioned frozen gates (superseded; original verdicts preserved): #57

Historical BF16 preparation: #54

Deliverables

• Linked phase-1 acceptance Issue with frozen workload, numerical gates and resource budget before candidate experiments.
• Reviewed minimal external-command integration and meaningful CPU contract tests.
• Reproducible single-GPU reference/candidate validation workflow with download/integrity preflight.
• Evidence tree containing plan, results, patch and validator, with real execution status and verdict.
• Independent review and separately reported PR readiness, phase-1 acceptance and remaining engine work.

Initial status (historical)

At campaign opening: INCONCLUSIVE; model integrity, hardware qualification and model execution had not been verified. Mainline nano-vllm currently implements Qwen3, so an unchanged upstream checkout is not a working Qwen3.8 baseline. A trusted reference and a correctly adapted candidate must be identified explicitly.

Implementation progress

• Draft PR Add Qwen3.8 FP8 correctness and service qualification workflows #55 implements paired correctness trials, a trusted-reference self-check, native numerical smoke and a separate real-socket service qualification workflow. Current integrated control passed the complete ready lane and 34 service adapter tests; source and run identities are governed by acceptance: Qwen3.8-27B-FP8 single-H20 text engine and HTTP service — phase 1 #57.
• The independent native engine is in Add dedicated Qwen3.8 FP8 text execution and HTTP service jiangkuaixue123/nano-vllm#1. Public entrypoint dispatch and bounded HTTP/SSE service are implemented, including cancellation, disconnect handling, state release, shutdown and concurrent-request isolation. Full native CPU regression passed 138 tests; subsequent GPU-specific fixes have separate review and versioned qualification.
• Official FP8 checkpoint byte authentication and H20-3e/reference-kernel readiness completed. No duplicate model download is needed. CPU and readiness results are not model acceptance.
• Pre-experiment inputs were frozen and independently reviewed before GPU execution. Historical failures are retained. Native compilation/lowering and reference loading were repaired. The isolated packed-singleton candidate has passed all 18 original L0 cases with independent actual-machine-code review (v7 / Slurm 1463); it has no measured performance or final-service qualification. The fixed-normalization reference adapter passed CPU regression and synthetic GPU row-invariance checks but still failed all 12 complete model NRMSE comparisons (v8 / Slurm 1466). All top-1 agreements were 1.0, which does not override the unchanged numerical gate. The source/runtime checks completed and allocation is released. A bounded trace-only diagnosis is being prepared; native numerical comparison remains gated on trusted-reference qualification.
• The fixed-protocol real-model HTTP/SSE lifecycle scope completed PASS, with both paired repetitions, independent evidence review and released resources; exact evidence is recorded in acceptance: Qwen3.8-27B-FP8 single-H20 text engine and HTTP service — phase 1 #57. This does not establish model quality or performance. Remaining acceptance work is full-prefix/cached reference validation, reference/native numerical comparison and service qualification of the final selected native revision. The prototype L0 and earlier service PASS apply to different revisions and cannot be combined into overall acceptance. Overall verdict remains INCONCLUSIVE. Private versioned plan/results/patch/validator bundles exist; reviewed publication remains separate from execution.
• Both PRs remain draft; neither has been merged and no Issue is closed. Exact revisions, freeze manifests, attempts and scoped results are maintained in acceptance: Qwen3.8-27B-FP8 single-H20 text engine and HTTP service — phase 1 #57.

Accepted service expansion

The owner explicitly added the engine entrypoint, HTTP/streaming generation, cancellation, state cleanup and concurrent isolation to phase 1. #57 records these required functional service milestones. Unit tests do not substitute for real-model/socket validation; performance gains remain outside phase 1. Implementation, CPU checks and the fixed-protocol real-model service-functionality scope are complete. Numerical qualification remains outstanding; the service result cannot clear numerical failures.