Repository navigation
Comparison freshness gate, construct-only ratchet legs, async comparison refresh - #73
Merged
Merged
Conversation
The comparison's SampleRateTap rows were hand-typed under a date line, so the DspTap bump to the Helium Q15 dot (ef6fdc4) moved one of them — the M55 Q15 steady state, 1,163 -> 861 instructions/frame — and nothing failed. Now compare.yml harvests its CMP_COUNT lines with their provenance (tree SHA, DspTap pin, run, toolchains) into async/bench/compare_counts.json (scripts/harvest_compare.py) and uploads it; scripts/update_compare_docs.py derives COMPARISON.md's embedded tables and key figures from that record; and a compare-docs CI job fails on drift, as icount-docs does for the README tables. Seeded from today's run 37920542901 on fc2949e: every cell reproduces the 2026-09-25 tables except the one that moved. Footnote 2 and the construction paragraph are reworded around the generated figures (float is now cheaper than Q15 on the M55 by 5 %, not 42 %), and a doubled parenthetical is gone. Construct-only ratchet legs, after rational's: async construct_q15 and bridge construct_{up,down}_q15_eco are the Q15 workloads with their stream removed — the converter built and the same input fixture synthesized, then one block processed — gated beside the streaming workloads so stream minus construct is the loop alone. The fixture is in the construct leg deliberately (24,000 soft-double sin() calls on M33). Measured on all three targets; every existing baseline identical to the instruction. Shares of the Q15 totals: async M33 88.8 %, Hexagon 70.2 %, M55 17.9 %; bridge M33 78 %, Hexagon 55 %, M55 19 %. Cross-check: async M33 pipeline minus construct is 1,188/frame against the 2 s/4 s split's 1,138 — the 50 is the servo's one-time lock-in, which the split counts as setup. PERFORMANCE.md and bridge/PLAN.md (M7e) record the numbers; the README tables regenerate; the book's scenario counts follow. Verified: gcc -Werror build with the ratchet workloads, 279/279 tests; clang-format and clang-tidy clean on the changed units; three-target ratchet green locally against the committed files; update_compare_docs idempotent; the book builds with zero warnings; both workflows parse. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G3HxzEGiZp7jStoMuYNuBK
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G3HxzEGiZp7jStoMuYNuBK
This was referenced Oct 9, 2026
tap
pushed a commit
that referenced
this pull request
Oct 10, 2026
The comparison plan (#73, #75, #76, #77) and its follow-up (#78) closed; this is the plan for what the family's plans and ledgers still carry open, sequenced into twelve weeks in three blocks, with "production-ready" defined as a thirteen-row checklist per engine so the verdict can be checked rather than believed. Drafted and then audited by two independent adversarial reviews (one against the trees' facts and settled decisions, one against engineering judgement). The audit changed the draft materially: the book's bridge chapters are done (#68), so they are off the list; rational has no cross-engine validation by design (scipy and decimate.h are its legs); TSan runs async only; four host compilers and three QEMU targets, not the draft's counts; the +1.9 % M55 ratio23_q15_eco drift is ab8fea3's stated move, not a DspTap bump, and the chain rows sit +926/+1,102; the hardware block never ran bridge or rational on silicon; nothing builds with -fno-exceptions; TSan has never seen the converter with two threads; no footprint, stack or worst-single-call figure exists; Setup 1's "< 1 ppm" criterion tests the counting, not the servo, and cheap USB dongles may not own a crystal; the timestamp servo is a four-to-five-week extension, not an overload, and must not be frozen in the release that adds it; "C ABIs frozen" needs a reconciled NULL model, an extension rule, SOVERSION and a compat test. Section 7 holds the six decisions the maintainer takes before week 1, each with a recommendation. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G3HxzEGiZp7jStoMuYNuBK
tap
pushed a commit
that referenced
this pull request
Oct 10, 2026
The comparison plan (#73, #75, #76, #77) and its follow-up (#78) closed; this is the plan for what the family's plans and ledgers still carry open, sequenced into twelve weeks in three blocks, with "production-ready" defined as a thirteen-row checklist per engine so the verdict can be checked rather than believed. Drafted and then audited by two independent adversarial reviews (one against the trees' facts and settled decisions, one against engineering judgement). The audit changed the draft materially: the book's bridge chapters are done (#68), so they are off the list; rational has no cross-engine validation by design (scipy and decimate.h are its legs); TSan runs async only; four host compilers and three QEMU targets, not the draft's counts; the +1.9 % M55 ratio23_q15_eco drift is ab8fea3's stated move, not a DspTap bump, and the chain rows sit +926/+1,102; the hardware block never ran bridge or rational on silicon; nothing builds with -fno-exceptions; TSan has never seen the converter with two threads; no footprint, stack or worst-single-call figure exists; Setup 1's "< 1 ppm" criterion tests the counting, not the servo, and cheap USB dongles may not own a crystal; the timestamp servo is a four-to-five-week extension, not an overload, and must not be frozen in the release that adds it; "C ABIs frozen" needs a reconciled NULL model, an extension rule, SOVERSION and a compat test. Section 7 holds the six decisions the maintainer takes before week 1, each with a recommendation. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G3HxzEGiZp7jStoMuYNuBK
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
PR 1 of the comparison plan (the rest: competitor set, bridge, rational).
compare.ymlnow harvests its counts with provenance (tree SHA, DspTap pin, run ID, toolchains) intoasync/bench/compare_counts.json(scripts/harvest_compare.py) and uploads it.scripts/update_compare_docs.pyderivesCOMPARISON.md's embedded tables and key figures from that record, and a newcompare-docsCI job fails on drift, the same shape asicount-docs.compare.ymlrun 37920542901 onfc2949e(DspTapef6fdc4). Every cell reproduces the 2026-09-25 tables except the one the Helium Q15 dot moved: M55 Q15 steady state, 1,163 → 861. Footnote ² and the construction paragraph are reworded around the generated figures; a doubled parenthetical is gone.asyncconstruct_q15,bridgeconstruct_{up,down}_q15_eco. Each is the Q15 workload with its stream removed, the converter built and the same input fixture synthesized, then one block processed, so stream − construct is the loop alone. Baselines recorded on all three targets; README tables regenerated.Why
The SampleRateTap rows in
COMPARISON.mdwere hand-typed under a date line, so a DspTap bump that moved a count failed nothing: the Helium dot landed on 2026-09-30 and the doc stayed at the September numbers until this morning's run exposed it.The construct legs are the mitigation
PERFORMANCE.mdhas queued since July and bridge's M7e asked for. On the M33 the Q15 totals are mostly constructor (async 89 %, bridge 78 %), so a hot-path regression there was diluted several times over before the ±3 % gate saw it.Verification
main), not local. The derivation matches the hand tables cell for cell at the 2026-09-25 pin;update_compare_docs.pyis idempotent.pipeline_q15 − construct_q15on M33 is 1,188/frame against the 2 s/4 s split's 1,138. The 50 is the servo's one-time lock-in, which the split counts as setup; both are measured andPERFORMANCE.mdsays which answers which question.sin(), 52 M soft-double instructions on M33. Without it in the subtrahend, "stream − construct" was 1,729/frame, i.e. not the stream. Rational'sconstruct_q15_ecohas that gap; a follow-up could align it.mainis merged in. On the merged tree: no conflicts, the new retired-words gate reports nothing, both freshness gates regenerate with no diff, gcc-Werrorwith every engine's WERROR and the C ABIs on, 305/305 tests, book clean.style.yml). Both workflows parse.compare.ymlharvest step end to end on CI (it runs on dispatch only). The harvest script was exercised on run 21's log, which is the same input the step writes.Notes for the reviewer
async/bench/compare_counts.jsonis the comparison's equivalent ofbaselines.json: a measured record with provenance. Re-measure by dispatchingcompare.yml, downloading the artifact, committing it and the regenerated block.ef6fdc4; after The 2026-10 adversarial audit and its response: S0 to S5, family version 0.6.0 #72 the tree pinsd7ebe4b(thechain.hfix, rational only). The counts stand — nothing async or bridge compiles changed — and the gate deliberately does not require pin equality, so a bump that moves no kernel does not force a 10-minute dispatch. The provenance line says what was measured; the reader decides whether the pin since matters.PERFORMANCE.mdsays so.🤖 Generated with Claude Code
https://claude.ai/code/session_01G3HxzEGiZp7jStoMuYNuBK