From 2f616bebbc525c067ce5ed4a33c78ce813ae3087 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 22 Oct 2026 14:10:21 -0700 Subject: [PATCH 1/3] Mark P8's and R1's moved readings; give M2's #549 its head note's place - decisions/P8: the #383 + #479 entry's JUAN Y GARCIA 'J. G.' reading is marked superseded by #461 (decisions/R3); both views give 'J. Y. G.'. - decisions/R1: the #760 entry's nee example moved with #601 (1037c87c), measured either side; for 'Smith, ...' the comma was a family comma before as after. - decisions/M2: Not standing no longer lists the #538 resolution a later decision reversed; its head note carries that (README edit kind 1). Co-Authored-By: Claude Opus 5.5 --- docs/design/decisions/M2-maiden-marker-rule.md | 2 +- ...P8-single-letter-connective-against-the-initial-it-spells.md | 2 +- .../design/decisions/R1-fields-are-views-of-the-parsed-words.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/design/decisions/M2-maiden-marker-rule.md b/docs/design/decisions/M2-maiden-marker-rule.md index a8577ed3..0aade2a4 100644 --- a/docs/design/decisions/M2-maiden-marker-rule.md +++ b/docs/design/decisions/M2-maiden-marker-rule.md @@ -29,7 +29,7 @@ - Inside the clause S2's company reads as it does anywhere (`Jane Doe nee Smith PhD MEng` → maiden 'Smith', suffix 'PhD MEng'), and a marker behind a credential is an ordinary word (`Jones Jr. née Smith` → middle 'Jr. née' under the default order), which the credential's run takes in where the run starts (`Jane Doe Jr. nee Smith` → suffix 'Jr. nee Smith') — 2026-09-27 (Derek), #544; 2026-10-04 (Derek), #601. - Degenerate input gets no designed reading (garbage in, garbage out): `Berg, Jane van der nee Smith DO` → middle 'van der nee Smith', family 'Berg', suffix 'DO' is recorded so its move is not read as a regression — 2026-10-04 (Derek), #601. -Not standing: the deviation recorded rather than repaired by 2026-09-22 #397 follow-up, resolved by 2026-09-26 #538, a resolution itself reversed by 2026-10-03 #549; the #548 deferral of 2026-09-26 #535, closed into #601 and #602; the trailing-title follow-up drafted for Derek by 2026-09-19 (Derek), #533; #535 took it; the skeleton migration of 2026-10-19 #761, carried out. +Not standing: the deviation recorded rather than repaired by 2026-09-22 #397 follow-up, settled by 2026-10-03 #549 (the #538 entry it superseded is marked so in its head); the #548 deferral of 2026-09-26 #535, closed into #601 and #602; the trailing-title follow-up drafted for Derek by 2026-09-19 (Derek), #533; #535 took it; the skeleton migration of 2026-10-19 #761, carried out. Elsewhere: a maiden name not making H1 decline (`Dr. Jane née Smith` → family 'Jane') — decisions/H1's 2026-08-25 (#410) entry; a one-letter numeral needing words to spare in the clause-free name, which keeps the V of `John née Jones Smith V` in the maiden name — decisions/S10's 2026-10-17 (Derek), #718 entry; a marked clause in brackets, read whole as the maiden name under the default policy — decisions/M3; a comma after a marker inside a clause unmarking it, and the divergence between M3's whitespace test and tokenize living on through the nakaguro (`佐藤 花子 "旧姓・田中" (旧姓 鈴木)` → nickname '旧姓 田中', maiden '鈴木') — decisions/N1's 2026-10-12 #653 entry and decisions/M3; the lone name word left beside a maiden name being the family (`Jane née Jones Smith` → family 'Jane', maiden 'Jones Smith') — decisions/M4; a declared delimiter parting a trailing suffix part — decisions/C10's 2026-10-03 (Derek), #549 entry; the credential's trailing run taking a suffix word and everything behind it (`Jane van der Berg née Jr Jones` → 'van der Berg née' one name part, the family under the default order, suffix 'Jr Jones') — decisions/S8 (#602); a maiden clause's numeral not held to its capitals — decisions/T4's 2026-10-18 (Derek), #731 reverted entry; `roz` and Scandinavian `f.` kept out of the markers, now on the words (`uv run python tools/vocab/query.py --excluded birth_marker`) — decisions/vocabulary-exclusions. diff --git a/docs/design/decisions/P8-single-letter-connective-against-the-initial-it-spells.md b/docs/design/decisions/P8-single-letter-connective-against-the-initial-it-spells.md index 70450ddc..6211f6e0 100644 --- a/docs/design/decisions/P8-single-letter-connective-against-the-initial-it-spells.md +++ b/docs/design/decisions/P8-single-letter-connective-against-the-initial-it-spells.md @@ -17,7 +17,7 @@ Open: none in this file. - 2026-07-30 #267 — the four-word single-letter asymmetry: a bare Latin capital connective is vetoed as an initial while a Cyrillic capital joins. #267's closure ("v2.0 behaves this way by default … verify it was the right call") showed only the Cyrillic half; the Latin-capital veto was never separately adjudicated, which rules/P8 records as an Accepted consequence (rules/P3 until the 2026-10-19 split). -- 2026-09-13 #383 + #479 — BLESSED for the mixed-case half, and DECIDED BY VOCABULARY for the one-case half, which is the half the question never reached. A single-letter connective joins only on positive evidence. Where a name is written in more than one case the writing is that evidence and nothing moves: a bare LATIN capital reads as an initial ("Jose E Maria Santos" → middle "E Maria"), which is #383 answered with "bless". The shape is still LATIN and the phrase "Latin-capital veto" does not retire with this change — `_INITIAL` is ASCII-only, so a Cyrillic capital is never initial-shaped, joins, and #267's reading above is untouched. An earlier draft of this work claimed the phrase retires; it was overstated, and what became evidence-based is the one-case half alone. +- 2026-09-13 #383 + #479 (its `JUAN Y GARCIA` "J. G." reading in both views superseded by 2026-09-20 #461, decisions/R3: both give "J. Y. G.") — BLESSED for the mixed-case half, and DECIDED BY VOCABULARY for the one-case half, which is the half the question never reached. A single-letter connective joins only on positive evidence. Where a name is written in more than one case the writing is that evidence and nothing moves: a bare LATIN capital reads as an initial ("Jose E Maria Santos" → middle "E Maria"), which is #383 answered with "bless". The shape is still LATIN and the phrase "Latin-capital veto" does not retire with this change — `_INITIAL` is ASCII-only, so a Cyrillic capital is never initial-shaped, joins, and #267's reading above is untouched. An earlier draft of this work claimed the phrase retires; it was overstated, and what became evidence-based is the one-case half alone. WHERE THE WRITING SAYS NOTHING. A name written wholly in ONE case — all upper and all lower alike — carries no case evidence about any letter in it. The reading there comes from a new marker subset `Lexicon.conjunctions_ambiguous` ⊂ `conjunctions`, default `{e}`: a member reads as an INITIAL and reports the new `AmbiguityKind.CONJUNCTION_OR_INITIAL`, a non-member JOINS even written as a bare capital and reports nothing. That is the third instance of the ambiguous-subset pattern, after `particles_ambiguous` and `suffix_acronyms_ambiguous`, so no mechanisms.md entry is owed. ROW 1 OF #479 IS THE DEFECT, NOT ROW 4 (Derek, 2026-09-12). The one-case lowercase reading is the one that was wrong, not the one-case uppercase reading that looked wrong beside it. WHY `e` AND NOT `y`. A bare E initial is common (Edward, Elizabeth) and an "e" between two surnames is rare outside couple listings; a bare Y initial is rare and "y" between two surnames is the commonest Hispanic compound and this library's oldest fixture ("Velasquez y Garcia"). The 2026-08-18 LOCALE CONTROL bullet under #3-0-reevaluations already said that which letters a tradition wants joined is per-word POLICY rather than membership; this subset is that policy made configurable, which is the smaller half of what that bullet asked for and ships without locale packs. `i` is not here because it is not conjunction vocabulary at all yet; it ships in the subset when #397 adds it, a bare I initial being as common as a bare E. diff --git a/docs/design/decisions/R1-fields-are-views-of-the-parsed-words.md b/docs/design/decisions/R1-fields-are-views-of-the-parsed-words.md index d4b4441f..8dac9d86 100644 --- a/docs/design/decisions/R1-fields-are-views-of-the-parsed-words.md +++ b/docs/design/decisions/R1-fields-are-views-of-the-parsed-words.md @@ -39,5 +39,5 @@ Open: none in this file. DECLINED, all four with the evidence: marking the boundary at the core-drop site (#437's MARK-DONT-STRIP shape) — `dropped` already holds the fact with its span, and a second recording of it is the duplication that mechanism exists to prevent, one level up; making the `"joined"` tag role-aware — within a piece it is role-blind and correct for every role, `Smith, Ph. D. Smith` giving `first_list == ['Ph. D.']`, and only the between-piece half was ever a suffix concept; a render-time span scan instead of the recorded tag — `_facade.__setstate__` and `ParsedName.replace()` synthesise span-less tokens, so an unpickled name has nothing to scan and the tag IS the entry structure a pickle carries; and fixing the no-comma path inside group's block as a third branch beside `tail` and `reading`, which is the shape #429 took. - 2026-09-06 #511 — a suffix value handed to `revise()` derives its entries from its own commas, by the rule a whole name uses. `Parser.revise` sub-parsed each value and forced every harvested token to the named role AFTER the sub-parse had run R1's entry pass, which keys on Role.SUFFIX; a bare 'MD PhD' reads there as a title and a family name, so the pass joined nothing and the field rendered 'MD, PhD'. The fix runs the same pass again: `revise` now sub-parses to a ParseState, forces the role on every non-dropped token, and calls `suffix_entries` — the pass, lifted unedited out of post_rules' tail into a function the two callers share (mechanisms.md#ONE-PREDICATE-PER-QUESTION) — over the forced state, then assembles and harvests as before. So a comma in the value parts two credentials and a space joins them. TWO SPELLINGS of the lifted pass, and the reason is the call budget: a first draft had post_rules call the state-in/state-out wrapper, and the second ParseState build cost three more calls per parse against the band tests/v2/test_benchmark.py holds (py3.11, 2026-09-06, tools/perf/call_count.py: 450 calls/name before the move, 451 with the in-place worker `_mark_suffix_entries` that post_rules now calls, 454 with the draft; the facade band tops at 455.9), so the worker writes in place as every other post rule does and the wrapper exists for the one caller with no token list of its own. NO `_run` HELPER for the same reason: a draft routed `parse()` through a private state builder shared with `revise`, and that cost one frame per parse on the hot path (py3.11, 2026-09-06: 415 calls/name against 414 without it, in a 402-418 band), so the four-line construction is spelled twice and `parse()` is untouched. MEASURED 2026-09-06 over the 1117 distinct corpus names, revising each of the 368 with a non-empty suffix by its own suffix: 38 suffixes moved before and 1 after, and the differential gate was byte-identical at all four baselines (recipe in the #511 issue body). THE ONE LEFT is '김민준씨, J.씨', and it is not entry structure: the whole-name parse keeps 'J.씨' one glued suffix token, suffix '씨, J.씨', while the sub-parse of the bare value '씨, J.씨' peels the honorific off the initial, so the revised field renders '씨, J. 씨' — right entries, the spurious comma of before ('씨, J., 씨') gone, one word read differently by the value's own parse than by the whole name's. That is the "classified ON ITS OWN" limit `revise`'s docstring has always recorded, and CJK honorific peeling is a W-rule question this change does not move; pinned in `test_revise_reads_a_glued_honorific_on_its_own`. SUPERSEDES the phd-merge acceptance of 'Ph., D.' on this path (its bullet says how). STALE TAGS, tried and backed out: a draft cleared the sub-parse's own "joined" with the forcing and re-derived it, because the pass only ADDS the tag and `revise(n, family="Jones MD PhD")` carries the sub-parse's between-piece suffix mark on 'PhD' onto a FAMILY token. Measured 2026-09-06 against `330ee55`, the clear also destroyed every WITHIN-piece mark on a non-suffix value — 'D.' of `revise(n, family="John Ph. D. Smith")` lost the merge mark the #436 bullet's DECLINED list calls role-blind and correct for every role — while on a ParsedName only the suffix string view reads "joined" (`_text_for`'s suffix_join gate) — the facade's `_list_for` heals it for every role, but no path puts a revised name into a HumanName, the v1 setters going through `replace()`, so wiring those setters onto `revise` is the change that would show the stale mark, as `last_list == ['Jones', 'MD PhD']` — `initials()` and every field string being identical at both trees for every shape measured. So the tags are kept minus FOLDED_TAG as before; the between-piece mark on a forced non-suffix role is a tag-only oddity that predates this change and stays, and for a suffix value nothing depends on the sub-parse's marks, every pair it joined sharing a bucket with no parting token and the pass setting it again; pinned in `test_revise_keeps_the_sub_parses_within_piece_mark`. Dropped tokens keep their role and tags through the forcing, because the pass filters them by index and assemble omits them. ONE MORE LIMIT, pinned in `test_revise_leaves_a_policy_delimiter_unparted_without_a_tail_segment`, and it is the "classified ON ITS OWN" limit again rather than a rule of revise's: a delimiter the policy names through `extra_suffix_delimiters` is dropped, and so parts entries, only on a segment after a comma that the reading of the words makes a tail, and a value with no comma of its own has none — under `Policy(extra_suffix_delimiters=frozenset({" - "}))` the whole name 'Doe, John, MD PhD - FACS' renders 'MD PhD, FACS' while `revise(n, suffix="MD PhD - FACS")` renders 'MD PhD - FACS', the dash surviving as a token and the forced role making it a suffix word; measured 2026-09-06, a value whose own words read with a tail segment does part ('John Doe, MD - FACS' revises to 'John Doe, MD, FACS') and one after a suffix comma does not ('MD, PhD - FACS' stays), which is how the value's words read as a name deciding it. The round-trip is unaffected, the whole-name view having already rendered that boundary as a comma. DECLINED: a list-valued `revise(p, suffix=["MD PhD", "FACS"])`, which widens the API and leaves the string path where it was; documenting the limit with pins alone, the docstring already recording it and #511's measurement showing it reachable on every space-joined run #436 produced; and a text-only read of the delimiter cores inside `revise`, a second rule for a case no round-trip reaches. Out of scope and left as is: `ParsedName.replace(suffix="MD, PhD")` renders 'MD,, PhD', `replace` whitespace-splitting by contract and the facade setters riding on it for v1 parity. - 2026-10-18 #728 — MOVED FROM decisions/C1. These entries decide that one comma segment is one suffix entry, which is R1's entry pass rather than the comma decision, and were filed under decisions/C1 until #728; they moved unchanged but for the pointers the move broke, the measurement paragraphs, each cut to one line naming its population, counts, baseline and PR, and the superseded passages, each cut to its decision and what replaced it. -- 2026-10-19 #760 — MOVED FROM the entry pass's code comment (`_post_rules._mark_suffix_entries`), the facts the entries above did not hold. ORDER: the pass runs after assign because it keys on Role.SUFFIX; the same span rule run role-blind would join the A and B of `John A B Smith` into one middle_list element, pinned by `test_the_pass_runs_after_roles_are_settled` in tests/v2/pipeline/test_post_rules.py. Within post_rules it runs last by convention, not by necessity: every retag there targets a name role (`_retag` is called with FAMILY, GIVEN, MIDDLE and `_name_positions`' return, which is those three, and the `role=Role.FAMILY` replaces of P6's no-comma attachment and O3's fold are FAMILY too), so no rule moves a token into or out of SUFFIX or {TITLE, NICKNAME, MAIDEN} and the predicate reads the same answer wherever in the stage it stands (measured 2026-09-06 by reading the stage's retag targets). THE UNGATED CORE READ: group drops a core only on a `tail` segment, while the pass reads `delimiter_cores` whole and asks by TEXT alone, so a dropped token whose text the policy names parts the run whatever dropped it. The gate is not needed: a core group dropped was on a tail segment by construction, and the one case the ungated read adds is a maiden marker the policy also lists — under `Policy(extra_suffix_delimiters=frozenset({" nee "}))`, `John Doe, MD nee Jones PhD` renders 'MD, PhD' where the default policy renders 'MD PhD' (measured 2026-09-06), the writer's configuration having named that text a separator. That example no longer reaches the case: re-measured 2026-10-19 while moving it here, `John Doe, MD nee Jones PhD` reads title 'MD', given 'nee', middle 'Jones', family 'John Doe' and suffix 'PhD' under both policies, the comma now read as a family comma, so the marker is never dropped and the pass sees one post-nominal; `Smith, MD nee Jones PhD`, the bare-marker example of the #436/#437 entry above, reads the same way. The witness today is the bracketed clause: `Smith, MD (nee Jones) PhD` renders 'MD PhD' under the default policy and 'MD, PhD' under that one, extract having dropped the marker as Role.MAIDEN and group never dropping it, so the ungated text read is what parts the run (re-measured 2026-10-19 in #760's review). The same input is the filter's example below. THE DROPPED-INDEX FILTER is belt-and-braces: cores leave `pieces` before assign and get no role, and the one dropped token that arrives roled is the MAIDEN one tokenize roles from an extracted clause (`Smith, MD (nee Jones) PhD` drops index 2, the marker, as Role.MAIDEN); the filter keeps the suffix list and the parting scan agreeing about what a dropped index is. +- 2026-10-19 #760 — MOVED FROM the entry pass's code comment (`_post_rules._mark_suffix_entries`), the facts the entries above did not hold. ORDER: the pass runs after assign because it keys on Role.SUFFIX; the same span rule run role-blind would join the A and B of `John A B Smith` into one middle_list element, pinned by `test_the_pass_runs_after_roles_are_settled` in tests/v2/pipeline/test_post_rules.py. Within post_rules it runs last by convention, not by necessity: every retag there targets a name role (`_retag` is called with FAMILY, GIVEN, MIDDLE and `_name_positions`' return, which is those three, and the `role=Role.FAMILY` replaces of P6's no-comma attachment and O3's fold are FAMILY too), so no rule moves a token into or out of SUFFIX or {TITLE, NICKNAME, MAIDEN} and the predicate reads the same answer wherever in the stage it stands (measured 2026-09-06 by reading the stage's retag targets). THE UNGATED CORE READ: group drops a core only on a `tail` segment, while the pass reads `delimiter_cores` whole and asks by TEXT alone, so a dropped token whose text the policy names parts the run whatever dropped it. The gate is not needed: a core group dropped was on a tail segment by construction, and the one case the ungated read adds is a maiden marker the policy also lists — under `Policy(extra_suffix_delimiters=frozenset({" nee "}))`, `John Doe, MD nee Jones PhD` renders 'MD, PhD' where the default policy renders 'MD PhD' (measured 2026-09-06), the writer's configuration having named that text a separator. That example no longer reaches the case: re-measured 2026-10-19 while moving it here, `John Doe, MD nee Jones PhD` reads title 'MD', given 'nee', middle 'Jones', family 'John Doe' and suffix 'PhD' under both policies, the comma now read as a family comma, so the marker is never dropped and the pass sees one post-nominal; `Smith, MD nee Jones PhD`, the bare-marker example of the #436/#437 entry above, reads the same way. (Corrected 2026-10-22, #762: both moved with #601, measured either side of 1037c87c, where before it they read maiden 'Jones' with suffix 'MD PhD'; what moved is the marker, which no longer takes a clause in that part, and for `Smith, …` the comma was a family comma before as after.) The witness today is the bracketed clause: `Smith, MD (nee Jones) PhD` renders 'MD PhD' under the default policy and 'MD, PhD' under that one, extract having dropped the marker as Role.MAIDEN and group never dropping it, so the ungated text read is what parts the run (re-measured 2026-10-19 in #760's review). The same input is the filter's example below. THE DROPPED-INDEX FILTER is belt-and-braces: cores leave `pieces` before assign and get no role, and the one dropped token that arrives roled is the MAIDEN one tokenize roles from an extracted clause (`Smith, MD (nee Jones) PhD` drops index 2, the marker, as Role.MAIDEN); the filter keeps the suffix list and the parting scan agreeing about what a dropped index is. - 2026-10-20 #761 — MOVED FROM rules/R1's revise() Accepted clause when R1 moved to the skeleton format, the measurement it carried: the #511 bullet counted 368 of 1117 distinct corpus names with a suffix on 2026-09-06; re-measured 2026-09-10 with that bundle's rows in the corpora, 372 of 1156 distinct names, and the same single failure, '김민준씨, J.씨'. From da3e8551a26cfe0fbc022320c193b9547131c45c Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 22 Oct 2026 14:17:38 -0700 Subject: [PATCH 2/3] Close #762: retire the Standing exempt list; fix the round-3 leftovers Every decisions file now opens with ## Standing except the five reference lists, so the exempt list and its frozen original set go: tests/v2/decisions_without_standing.json is deleted and REFERENCE_LISTS, its size asserted, fixes the one kind of file that carries no block. standing_roster_findings and its tests follow, and decisions/README.md and docs/design/AGENTS.md say so. Round-3 review leftovers (#830, #831, #832): - rules/R3: a base letter and its mark initial whole only where they compose; a mark NFC leaves separate (a nukta after ja, U+095B being a composition exclusion) is left out, as decisions/R3's #585 decided. - decisions/parse-cost: a 2026-10-04 #601 entry records the link pin's re-pointing (2,301 -> 2,678); test_benchmark's _CALL_BASELINE comment names #624 and #625. - decisions/cjk-full-stops: the bullet approved 2026-09-09 gets its own dated line, and its two Standing lines point there. - decisions/README.md: #832's bundle-dating sentence is split into four. Co-Authored-By: Claude Opus 5.5 --- docs/design/AGENTS.md | 2 +- docs/design/decisions/README.md | 4 +- docs/design/decisions/cjk-full-stops.md | 5 +- docs/design/decisions/parse-cost.md | 3 +- .../rules/R3-initials-honor-the-fold.md | 2 +- tests/v2/decisions_without_standing.json | 4 - tests/v2/design_docs.py | 89 +++++------------ tests/v2/rules_budget.json | 2 +- tests/v2/test_benchmark.py | 3 +- tests/v2/test_design_docs.py | 99 ++++++++----------- 10 files changed, 74 insertions(+), 139 deletions(-) delete mode 100644 tests/v2/decisions_without_standing.json diff --git a/docs/design/AGENTS.md b/docs/design/AGENTS.md index f2d8c02b..a9dceb47 100644 --- a/docs/design/AGENTS.md +++ b/docs/design/AGENTS.md @@ -22,7 +22,7 @@ Two shapes, needing different instruments. CROSS-SESSION distillation is what th **Reviewing a docs/design change: the axes.** Run these before merging anything that touches rules/, decisions/, mechanisms.md — or THIS FILE, which carries the conventions those three rest on and has the same failure modes. Axes 3, 5, 6 and 12 are rules-specific (they need rule statements, examples and the rules a rule names); the other eight apply to any of the four — axis 11 included, since an invariant over two inputs is prompted by what a rule implies but is RUN without reading one, which is the whole of its value. Say which axes you ran and which you skipped as inapplicable — a checklist that asks for something meaningless teaches the reader to skim it. Axes 1–9 were each earned by a defect that survived every other check on #386, where the review passes surfaced ten issues in total — nine of them by these axes, one by the spec self-review above — and unaided prose reading surfaced none. Axes 10 and 11 were earned the same way on other arcs — #291's, which PREDATES this list, and #461/#466's, which follows it. Axis 12 was earned by no defect at all, which is its point: it covers the growth every other axis passes. The parenthetical is the case that earned the axis. -Skip what the suite already enforces. `tests/v2/test_rules_doc.py`, `tests/v2/test_doc_citations.py` and `tests/v2/test_design_docs.py` machine-check example lines and their `deviates:` today-values, boundaries, the one-of `implemented:`/`tracked:` rule, `implemented:` against the modules that actually cite the rule, the rule IDs a rule names anywhere in its prose, cross-doc anchor IDs, citation excerpts quoted verbatim, legacy citation forms (the pre-#713 file-anchor form among them), that every rule and decisions file is indexed once, under a title matching its heading and file name, each decisions file's `## Standing` block against its live entries and against the roster of files exempt from one and the index's Reference lists, a rule file reusing an ID a decisions file records `Retired:` and a misspelled `Retired:` line, the README's copy of rules/D2 against the file and its split-candidate grep against the rules' Signal lines, that the NFD-typed examples stay decomposed, a denylist of spellings that shipped wrong (`tests/v2/test_doc_spellings.py`, over nameparser/ and docs/), each rule's `Signal:` line (its presence, spelling and place) and the README's list of the tags those lines use, a hub's rules against the bullets under its Then, both ways, a comment naming an Except or Precedence bullet against that bullet's bold lead, the skeleton grammar in `tests/v2/test_rules_skeleton.py` (labels, field order, fences, the footer, one paragraph each for When, Tolerated and Then, the two sentence limits — Then one sentence, Because three at most — and an example under every Except bullet), the refusal of a rule in the retired block format, and — in `tests/v2/test_rules_budget.py` — each rule's recorded size. Assume they pass; a finding one of them would catch is a false positive. Read those files rather than trusting this sentence — it named five of the nine guards when written, and goes stale as guards are added. +Skip what the suite already enforces. `tests/v2/test_rules_doc.py`, `tests/v2/test_doc_citations.py` and `tests/v2/test_design_docs.py` machine-check example lines and their `deviates:` today-values, boundaries, the one-of `implemented:`/`tracked:` rule, `implemented:` against the modules that actually cite the rule, the rule IDs a rule names anywhere in its prose, cross-doc anchor IDs, citation excerpts quoted verbatim, legacy citation forms (the pre-#713 file-anchor form among them), that every rule and decisions file is indexed once, under a title matching its heading and file name, each decisions file's `## Standing` block against its live entries and against the index's Reference lists, the one kind of file that carries none, a rule file reusing an ID a decisions file records `Retired:` and a misspelled `Retired:` line, the README's copy of rules/D2 against the file and its split-candidate grep against the rules' Signal lines, that the NFD-typed examples stay decomposed, a denylist of spellings that shipped wrong (`tests/v2/test_doc_spellings.py`, over nameparser/ and docs/), each rule's `Signal:` line (its presence, spelling and place) and the README's list of the tags those lines use, a hub's rules against the bullets under its Then, both ways, a comment naming an Except or Precedence bullet against that bullet's bold lead, the skeleton grammar in `tests/v2/test_rules_skeleton.py` (labels, field order, fences, the footer, one paragraph each for When, Tolerated and Then, the two sentence limits — Then one sentence, Because three at most — and an example under every Except bullet), the refusal of a rule in the retired block format, and — in `tests/v2/test_rules_budget.py` — each rule's recorded size. Assume they pass; a finding one of them would catch is a false positive. Read those files rather than trusting this sentence — it named five of the nine guards when written, and goes stale as guards are added. 1. **Recompute every number.** Any count, ratio or "N names" in the diff gets recomputed by a script written now, never read. (A filter described as returning one name returned three; a rule's stated blast radius listed two names its own scope excluded.) 2. **Your detector is a second, unreviewed implementation.** It must read the same vocabulary and boundaries the rule reads. (A particle-run detector that walked only the never-given half — the rule chains through ANY particle — reported 50 false movers where the true count was 1, printing cleanly both times.) diff --git a/docs/design/decisions/README.md b/docs/design/decisions/README.md index 520e4a5e..6d8cbab3 100644 --- a/docs/design/decisions/README.md +++ b/docs/design/decisions/README.md @@ -11,7 +11,7 @@ Entry conventions: 2. **A factual error is corrected in place**, with a note saying what was wrong — a date, an attribution, a measurement or a pointer. decisions/C1's 2026-07-12/13 entry, whose parenthetical moved a misdated credit, is the shape. 3. **Compaction, on a stated trigger.** A superseded entry is cut to a tombstone: its date, issue, verdict line and `SUPERSEDED by`, plus whatever it recorded that the superseding entry did not replace — a gap it found standing before its change, or an option it declined that the reversal does not answer, stays beneath the tombstone (decisions/T4's #731); a superseded passage inside a live entry is cut to its decision and what replaced it. A measurement paragraph (what moves, cost, blast radius) is cut to one line naming its population, counts, baseline and PR once its change has shipped in a release. The full text stays in git and in the PR the entry cites. #728 is the worked compaction. A harvested entry (below) keeps its framing line through compaction, a tombstone included, and its measured-vs-remembered markings wherever its measurements survive. A trap #759's review found: a one-line replacement carries each count with the classification the old text gave that count and no other (decisions/S2's #614 line had given a whole grid the classification measured on one slice of it). 4. **A per-word fact that moves into the vocabulary's TOML is REPLACED by a query that finds it** (`uv run python tools/vocab/query.py --issue 661`, `--excluded generation`), not left beside a dated "moved" note, which would keep two copies to drift. Only the per-word sentences go; the decision, its criterion and its measurement stay as written. Sanctioned since 2026-10-18 (decisions/vocabulary-model). -- **`## Standing`**, at the head of a file: the one part edited freely. One line per decision in force, in the present tense, ending with the entry that decided it by its exact date and the first issue its head names (`— 2026-10-06 #613`), or by its date alone where the head names none; where several entries share that date and issue, the pointer goes on with the words its entry's head continues with (`— 2026-09-14 (Derek)`, `— 2026-10-18 #718 review`), the longest such continuation winning and a date inside those words being part of them; a pointer with no words names the entry whose head ends there, and one whose words continue no head there fails, since nothing tells a typo from prose, so prose after such a pointer goes after a `.` or `;` (the words end there, or where the next pointer's date begins); where two heads are the same words, the pointer goes on with the words its entry's text opens with (`— 2026-09-13 #383/#479, AMENDS`), a key rather than prose, and so it does where another head of that date and issue continues past those words (decisions/M3's `— 2026-08-26 #335, the MECHANISM`, beside its `#335 (M3, …)` head); and a pointer still naming more than one entry fails as ambiguous (a parenthesized supersession note in a head is not read), plus a `Not standing:` line for entries whose plan was carried out or whose schedule events overtook, its pointers read and failing as a Standing line's do except that they may name a superseded entry — a decision reversed by a later DECISION is marked under 1 instead and an `Open:` line for each open question. It indexes the log and is never a decision itself: a Standing line with no entry behind it, or only a superseded one, fails `tests/v2/test_design_docs.py::test_every_standing_line_points_at_a_live_entry`, and changing a decision is still a new entry, which updates Standing in the same diff. The entries follow under `## Log`. An entry is a log bullet opening with its date, bare or in bold (`- **2026-10-14 #679 — …**`); a file's title founds no entry, so a bundle recorded as undated bullets under a dated title gains, when its block is written, one dated line opening its Log and naming the bundle, its verdict the title's words (`- 2026-09-10 #322/#323 — WHICH CHARACTERS ARE A PERIOD ON CJK TEXT: the bullets below.`). It dates what the title already dates, so it adds no decision and edits no entry. A file with no dated title dates its bundle line by the date its preamble gives the bullets. A bullet the bundle line's date does not fit, because its own text dates its decision otherwise (`- (A) 2026-08-17 — …`, `Amended 2026-08-23`; a date it gives for something else, such as when a code change landed, is not one) or, where its text dates none, because `git log -S` over `docs/design/` (not over the file, which every bullet entered on 2026-10-17, when #713 split the one decisions.md into these files) shows it entering on a later day than the bundle was written, its earliest hit (`--reverse`) on a phrase short enough to have stood on one line before the 2026-08-16 unwrap, 5e2ae60b, a first hit there meaning the phrase crossed a wrapped line (decisions/3-0-reevaluations' legacy-pickle bullet, added the day after its list was promoted; a bullet written the day its bundle was, as decisions/comma-suffix-arc's Declined bullets were, keeps the bundle line's date even where that date is older), gains a dated line of its own beside the bundle line, dated by that decision date or by the author date of the commit that added it, naming the bullet by the words it opens with, since a pointer reads only a bullet that opens with its date (decisions/3-0-reevaluations and decisions/comma-suffix-arc, 2026-10-22). Where a later decision reversed part of the bundle, its head says which, under edit kind 1. That note is for the reader: a pointer never reads a parenthesized note containing "superseded" (edit kind 1's wording), so a Standing line resting on a bundle is checked by hand against what the bundle's later entries and other files reversed, as for any entry superseded in part. Every decisions file created after C1's block landed (2026-10-19) opens with one, the cheap moment to write it being the file's first entry; the older files are listed in `tests/v2/decisions_without_standing.json`, decisions/C12 and decisions/vocabulary-model among them though they too were created that day, before the rule. `test_a_decisions_file_off_the_list_opens_with_standing` fails on a file off that list with no block (the heading on a line of its own; the words quoted in prose do not count), on a listed file that has gained one, on a listed key no file carries, and on a listed key the list did not hold when the rule landed (a fixed set in `tests/v2/design_docs.py`, its size asserted), so the list never holds a key it did not start with: a new file can join it neither alone nor in exchange for one that leaves. The block stands ahead of every `## Log` heading; a `## Log` before it is a finding, and so is a heading that misspells `## Standing` (a trailing space, a different level), which would otherwise switch the pointer checks off. A decision of another rule's that this log is the record of stays a bullet naming its rule (decisions/C13's line for rules/C2, decisions/P9's for rules/S5); one recorded under another key is an `Elsewhere:` line pointing at that entry, not a bullet. Whether the older files get one is #762. A file of Declined, Excluded, Open, removal or mapping lines is a reference list instead, indexed under the Reference lists heading at the end of the index: it carries no block. A listed file carrying a block, taking a log's shape (a dated entry, a bullet opening with a tag and a date as `- (A) 2026-08-17 — …`, or a dated title) or sitting on the exempt list is a finding, as is a listed key the exempt list did not hold when the rule landed (2026-10-21, #762). Whether a file in none of those shapes is a list or a log is a judgment the check cannot make, made in review. 3-0-reevaluations is not one: other files amend its lines without its saying so (decisions/cjk-comma-demotion scopes its FAMILY_COMMA line), the shape a Standing block's `Not standing:` and `Elsewhere:` lines are for. Five files split out on 2026-10-19 (C13, C14, P8, P9, T5) showed the rule was needed: none had a block, though C1's had landed the same day. **Before writing a line, look for what reversed it elsewhere.** A decision can be overturned by an entry in another file, which the log it stands in never mentions; #776 found two lines wrong that way, one while writing and one in review (P9's #578 note, reversed by decisions/S8's #602; P8's subset, widened by decisions/P3's #397), and #762's first batch (2026-10-21) found five entries whose heads did not say what had replaced part of them (decisions/S6's accepted `abdul Smith Jr Ma`, reversed by decisions/S7's #544; decisions/S4's `Doe, Jane nee Smith MA`, by decisions/M2's #601; decisions/H3's `Dr Smith V`, by decisions/S10's #718, caught only in that batch's PR review; decisions/P5's view comparison and decisions/S10's one-case `v`, by their own files' #614 and #721). So grep the whole folder for each issue and example a line cites, parse each example on the tree, and where a later decision reversed part of an entry mark that entry's head under 1, so the log says what Standing says. +- **`## Standing`**, at the head of a file: the one part edited freely. One line per decision in force, in the present tense, ending with the entry that decided it by its exact date and the first issue its head names (`— 2026-10-06 #613`), or by its date alone where the head names none; where several entries share that date and issue, the pointer goes on with the words its entry's head continues with (`— 2026-09-14 (Derek)`, `— 2026-10-18 #718 review`), the longest such continuation winning and a date inside those words being part of them; a pointer with no words names the entry whose head ends there, and one whose words continue no head there fails, since nothing tells a typo from prose, so prose after such a pointer goes after a `.` or `;` (the words end there, or where the next pointer's date begins); where two heads are the same words, the pointer goes on with the words its entry's text opens with (`— 2026-09-13 #383/#479, AMENDS`), a key rather than prose, and so it does where another head of that date and issue continues past those words (decisions/M3's `— 2026-08-26 #335, the MECHANISM`, beside its `#335 (M3, …)` head); and a pointer still naming more than one entry fails as ambiguous (a parenthesized supersession note in a head is not read), plus a `Not standing:` line for entries whose plan was carried out or whose schedule events overtook, its pointers read and failing as a Standing line's do except that they may name a superseded entry — a decision reversed by a later DECISION is marked under 1 instead and an `Open:` line for each open question. It indexes the log and is never a decision itself: a Standing line with no entry behind it, or only a superseded one, fails `tests/v2/test_design_docs.py::test_every_standing_line_points_at_a_live_entry`, and changing a decision is still a new entry, which updates Standing in the same diff. The entries follow under `## Log`. An entry is a log bullet opening with its date, bare or in bold (`- **2026-10-14 #679 — …**`); a file's title founds no entry, so a bundle recorded as undated bullets under a dated title gains, when its block is written, one dated line opening its Log and naming the bundle, its verdict the title's words (`- 2026-09-10 #322/#323 — WHICH CHARACTERS ARE A PERIOD ON CJK TEXT: the bullets below.`). It dates what the title already dates, so it adds no decision and edits no entry. A file with no dated title dates its bundle line by the date its preamble gives the bullets. A bullet decided on another day gains a dated line of its own beside the bundle line, naming the bullet by the words it opens with, since a pointer reads only a bullet that opens with its date (decisions/3-0-reevaluations, decisions/comma-suffix-arc, decisions/cjk-full-stops). The date is the decision's: the one the bullet's own text gives it (`- (A) 2026-08-17 — …`, `Amended 2026-08-23`; a date given for something else, such as when code landed, is not one), or, where it gives none, the author date of the commit that added the bullet. Find that commit with `git log -S --reverse` over `docs/design/`, not over the file, which every bullet entered on 2026-10-17 when #713 split decisions.md, and search a phrase short enough to have fit one line before the 2026-08-16 unwrap (5e2ae60b), a first hit there meaning the phrase crossed a wrapped line. A bullet written the day its bundle was keeps the bundle line's date, even where that date is older (decisions/comma-suffix-arc's Declined bullets). Where a later decision reversed part of the bundle, its head says which, under edit kind 1. That note is for the reader: a pointer never reads a parenthesized note containing "superseded" (edit kind 1's wording), so a Standing line resting on a bundle is checked by hand against what the bundle's later entries and other files reversed, as for any entry superseded in part. Every decisions file opens with one except the reference lists (below), a new file's cheapest moment to write it being its first entry; the files that predated the rule (C1's block landed 2026-10-19) were given theirs under #762, finished 2026-10-22, and the exempt list that held them while it ran is retired. `test_every_decisions_file_opens_with_standing_or_is_a_reference_list` fails on a file with no block that is not a reference list (the heading on a line of its own; the words quoted in prose do not count). The block stands ahead of every `## Log` heading; a `## Log` before it is a finding, and so is a heading that misspells `## Standing` (a trailing space, a different level), which would otherwise switch the pointer checks off. A decision of another rule's that this log is the record of stays a bullet naming its rule (decisions/C13's line for rules/C2, decisions/P9's for rules/S5); one recorded under another key is an `Elsewhere:` line pointing at that entry, not a bullet. A file of Declined, Excluded, Open, removal or mapping lines is a reference list instead, indexed under the Reference lists heading at the end of the index: it carries no block. A listed file carrying a block or taking a log's shape (a dated entry, a bullet opening with a tag and a date as `- (A) 2026-08-17 — …`, or a dated title) is a finding, as is a listed key outside the five fixed in `tests/v2/design_docs.py`'s `REFERENCE_LISTS` (2026-10-21, #762; fixed 2026-10-22). Whether a file in none of those shapes is a list or a log is a judgment the check cannot make, made in review. 3-0-reevaluations is not one: other files amend its lines without its saying so (decisions/cjk-comma-demotion scopes its FAMILY_COMMA line), the shape a Standing block's `Not standing:` and `Elsewhere:` lines are for. Five files split out on 2026-10-19 (C13, C14, P8, P9, T5) showed the rule was needed: none had a block, though C1's had landed the same day. **Before writing a line, look for what reversed it elsewhere.** A decision can be overturned by an entry in another file, which the log it stands in never mentions; #776 found two lines wrong that way, one while writing and one in review (P9's #578 note, reversed by decisions/S8's #602; P8's subset, widened by decisions/P3's #397), and #762's first batch (2026-10-21) found five entries whose heads did not say what had replaced part of them (decisions/S6's accepted `abdul Smith Jr Ma`, reversed by decisions/S7's #544; decisions/S4's `Doe, Jane nee Smith MA`, by decisions/M2's #601; decisions/H3's `Dr Smith V`, by decisions/S10's #718, caught only in that batch's PR review; decisions/P5's view comparison and decisions/S10's one-case `v`, by their own files' #614 and #721). So grep the whole folder for each issue and example a line cites, parse each example on the tree, and where a later decision reversed part of an entry mark that entry's head under 1, so the log says what Standing says. - **`Retired:`**, for a rule taken out of rules/: a line of its own in the rule's decisions file, opening `Retired:` and naming the date, the issue and what replaced the rule (`Retired: 2026-11-02 #800 — folded into C1.`), in the Standing block where the file has one, beside the dated entry that decided it. The ID is then never reused: `tests/v2/test_design_docs.py::test_no_rule_reuses_a_retired_id` refuses a rule file carrying it, a `Retired:` line in a file keyed by no rule ID, and a line that misspells `Retired:` (bold, a list marker or number, lowercase), which would switch the refusal off for its ID. No rule had been retired when the check landed (2026-10-20), so it guarded nothing on the tree until the first one is; its synthetic case is what shows it bites. - **Harvested entries** (content landed from a session's report) keep their provenance: a "measurements are that session's, spot-checked at landing" framing line, the contributor's measured-vs-remembered markings where they matter, and — the hard-won one — a flagged uncertainty is resolved by reading the source artifact, never by inference from the neighboring argument. - **`Declined:`** — proposals rejected WITH the evidence that killed them. Resolved-as-no is a decision; without a home for it, the next person re-derives the rejected proposal and its measurement. @@ -153,7 +153,7 @@ Entry conventions: ## Reference lists -Files of Declined, Excluded, Open, removal or mapping lines rather than a dated log, none of whose lines a later decision reverses without saying so on that line. Each line already says what stands, so they carry no Standing block (the entry conventions above). A file listed here takes no log's shape — no dated entry (a bullet opening with its date; a date inside a Declined or Excluded line is not one), no bullet opening with a tag and a date, no dated title — and only a file created before the Standing rule may be listed: a new decisions file opens with `## Standing` whatever it holds. +Files of Declined, Excluded, Open, removal or mapping lines rather than a dated log, none of whose lines a later decision reverses without saying so on that line. Each line already says what stands, so they carry no Standing block (the entry conventions above). A file listed here takes no log's shape — no dated entry (a bullet opening with its date; a date inside a Declined or Excluded line is not one), no bullet opening with a tag and a date, no dated title — and the five listed here are the only ones (`REFERENCE_LISTS`, its size asserted): a new decisions file opens with `## Standing` whatever it holds. - [api-churn-declines — do not re-propose without new cause](api-churn-declines.md) - [given-name-titles — deliberately unvalidated](given-name-titles.md) diff --git a/docs/design/decisions/cjk-full-stops.md b/docs/design/decisions/cjk-full-stops.md index 106729b2..617199f4 100644 --- a/docs/design/decisions/cjk-full-stops.md +++ b/docs/design/decisions/cjk-full-stops.md @@ -6,8 +6,8 @@ ONE GAP SEEN FROM TWO SIDES, which is why the two issues were answered together. - `_lexicon.FULL_STOPS` (`.．。｡`: the ASCII period and the fullwidth, ideographic and halfwidth ideographic stops) answers "which characters are a period" once, read by the lookup and storage fold, the script-classification fold, the surname site's head match, and the honorific peel's tail match, so `김민준 씨．`, `씨。` and `씨｡` each read suffix as `씨.` does — 2026-09-10 #322. - A stop glued to a name word is read at the edge of every site that reads the set, not only the lookup: `양. 지훈` reads given '지훈', family '양.', `양 지훈.` keeps its family-first order, `김. 민준` divides as family '김.', given '민준', and `김민준.` as family '김', given '민준.', the stop riding with the remainder — 2026-09-10 #322. -- A stop on a glued honorific rides with it: `田中さん.` reads family '田中', suffix 'さん.' and `김민준씨.` family '김', suffix '씨.', as their stop-less spellings do — 2026-09-10 #322. -- H2's opening-abbreviation shape declines a period-marked word carrying a script with no initials, so `田中.` reads family '田中.' and `田中. 太郎` family '田中.', given '太郎' while `Smith. John` and `Проф. Иванов` keep title; the veto is contains-any, so a word carrying such a character is no title even beside Latin letters — 2026-09-10 #322. +- A stop on a glued honorific rides with it: `田中さん.` reads family '田中', suffix 'さん.' and `김민준씨.` family '김', suffix '씨.', as their stop-less spellings do — 2026-09-09. +- H2's opening-abbreviation shape declines a period-marked word carrying a script with no initials, so `田中.` reads family '田中.' and `田中. 太郎` family '田中.', given '太郎' while `Smith. John` and `Проф. Иванов` keep title; the veto is contains-any, so a word carrying such a character is no title even beside Latin letters — 2026-09-09. - The fold is script-agnostic at the edge, so a Latin or Cyrillic word wearing a wide stop reaches the vocabulary that only the ASCII period reached (`Dr。 John Smith` and `Проф。 Иванов` read title, `John Smith V。` and `Smith, John V。` read suffix 'V。' where `Smith, John V.` keeps its middle initial); the four ASCII-only edge tests on Latin shapes (the initial `J.`, the bracketed credential `(Mgr.)`, the apostrophe rule, the trailing `V.` carve-out) stay ASCII, and a name with only interior wide stops (`Ｍ．Ｂ．Ａ．`) is untouched — 2026-09-10 #322. - `_normalize` composes NFC for a non-ASCII word at storage and at match time, which makes the fold symmetric, so NFD hangul, Latin and the other composable scripts match their vocabulary (`NFD("김민준, 씨.")` reads family '김민준', suffix '씨.'); the segmentation sites stay raw, so an NFD name goes unsplit rather than divided — 2026-09-10 #322. - NFKC was declined: it folds `．` and `｡` but not `。`, and at the storage fold it would rewrite stored vocabulary, which the `lower()`-not-`casefold()` decision already refused — 2026-09-10 #322. @@ -22,6 +22,7 @@ Open: none in this file. ## Log - 2026-09-10 #322/#323 — WHICH CHARACTERS ARE A PERIOD ON CJK TEXT: the bullets below. +- 2026-09-09 — TWO READINGS JOINED THE BUNDLE, approved in session the day before it landed: the bullet so headed below, the glued-period honorific (rules/W3) and H2's opening-abbreviation shape on a CJK word. - **FULL_STOPS names four characters; five sites read them.** `nameparser/_lexicon.py::FULL_STOPS` is `".．。｡"`: the ASCII period, FULLWIDTH FULL STOP U+FF0E, IDEOGRAPHIC FULL STOP U+3002 and HALFWIDTH IDEOGRAPHIC FULL STOP U+FF61. Four sites import the string, so "which characters are a period" is answered once: `_lexicon._normalize` (the lookup AND storage fold), `_vocab._normalized_for_script` (the script-classification fold), and in `_script_segment` the surname site's head match and the honorific peel's tail match. The FIFTH site reads no stop set at all. `_pieces.is_leading_title` — rules/H2's opening-abbreviation shape — refuses a period-marked word whose text carries a character of a script with no initials, through `_vocab.in_initialless_script` (renamed from `_in_initialless_script`; the same veto `is_initial` has carried since #320), so `_PERIOD_ABBREV` stays an ASCII-period pattern and the four-character set is not copied there. Not every period test reads this set, and the ones that do not are named so the claim is checkable: the interior-period shapes — the dotted acronym `M.A.` and the split `Ph. D.` (`_vocab`) — and FOUR ASCII-only edge tests on Latin shapes across THREE modules — the initial `J.` (`_vocab`), the bracketed credential `(Mgr.)` and the word-internal apostrophe rule (`_extract`), and the trailing middle-initial carve-out `V.` (`_assign`) — stay ASCII. What that does NOT mean is that the wider stops leave Latin text alone. The TESTS stay ASCII; the vocabulary FOLD does not, so a Latin word wearing a wide stop reaches vocabulary an ASCII-only carve-out would have kept it out of — `V。` folds to `v` and does not end in the ASCII period that `_reads_as_a_trailing_suffix`'s carve-out tests (`text.endswith(".")`, `_assign.py` ~line 479, whose docstring calls the period "the whole carve-out"), so the carve-out never fires for `V。` and it is roman five, suffix, where `V.` stays a middle initial through that same carve-out — not through `is_initial`, which is not the decider despite reading true on both spellings: `is_initial("V")` is True, yet `Smith, John V` (no period at all) still reads suffix `V`, so being initial-shaped never by itself keeps a word out of suffix. `Jr。` needs no carve-out argument and simply reaches its entry. An earlier wording of this passage called `Ｍ．Ｂ．Ａ．` and `V。` "a fullwidth-Latin question this set does not raise" (corrected 2026-09-10): that was right for `V。` alone. `V。` is an edge stop on a bare word, and the fold raises the question at every such edge, moving the parse as above. `Ｍ．Ｂ．Ａ．` does NOT move — every one of its periods is interior, none at an edge for `_lexicon._normalize`'s edge strip to reach, so the fold never touches it at all, and the dotted-acronym reading in `_vocab.suffix_as_written` (`n.replace(".", "")`) is ASCII-only regardless of the fold: `_normalize("Ｍ．Ｂ．Ａ．")` is `ｍ．ｂ．ａ`, which matches no acronym form the ASCII-only test recognizes. MEASURED 2026-09-10: `parse("John Smith Ｍ．Ｂ．Ａ．")` and `parse("John Smith, Ｍ．Ｂ．Ａ．")` are identical on `d37b8ec` and on this tree — `Ｍ．Ｂ．Ａ．` stays exactly as it was. The set raises the question at every EDGE fold, and the four ASCII-only tests are the places it is not raised rather than a boundary on the set. MEASURED 2026-09-10 on this tree: `Andrew Perkins (Mgr。)` reads nickname `Mgr。` where `Andrew Perkins (Mgr.)` reads title `Mgr.`; `Smith, John V。` reads suffix `V。` where `Smith, John V.` reads middle `V.`; and `John Smith, Jr。` reads suffix `Jr。` where `d37b8ec` read given `Jr。`, family `John Smith`. The last two are case rows now (`latin_roman_numeral_with_an_ideographic_full_stop`, `latin_suffix_with_an_ideographic_full_stop`), and `is_suffix_strict("Jr。", Lexicon.default())` and `_normalize("Dr。") == "dr"` pin the fold's Latin reach at unit level. Nothing rewrites token text at any of the five: the stop stays on the token it arrived with, and every parsed part is still an exact slice of the input (rules.md's T Background). - **NFC composes at LOOKUP, and that is what makes the fold symmetric.** `_normalize` now composes NFC for a non-ASCII word, at match time and at storage both. NFD hangul decomposes to jamo and missed every vocabulary entry, so `NFD("김민준, 씨.")` read title `씨.`, family `김민준` and now reads family `김민준`, suffix `씨.`. MEASURED 2026-09-09, re-measured 2026-09-10: no shipped entry changes under NFC — 0 of the 1735 strings in the twelve set-valued fields of `Lexicon.default()` (the standing pin `tests/v2/test_lexicon.py::test_every_shipped_entry_is_already_nfc` walks those twelve plus both halves of the five `capitalization_exceptions` pairs, 1745 strings, and asserts the same). RECOMPUTE: fold every string member of those fields through `unicodedata.normalize("NFC", w)` and count `!= w`. diff --git a/docs/design/decisions/parse-cost.md b/docs/design/decisions/parse-cost.md index 331dcac3..0d072437 100644 --- a/docs/design/decisions/parse-cost.md +++ b/docs/design/decisions/parse-cost.md @@ -29,10 +29,11 @@ Open: none in this file. ONE REPAIR WAS TRIED AND REVERTED: collapsing `post_rules`' three separate role-index scans into one pass. It saves **6 calls per parse** on 3.11 (the first draft said one, which is not even reachable — `_idx` is a one-line list comprehension, so each removed call frees two frames). Reverted because it was measured against a noisy timing harness that showed nothing; at 6 calls it is 9% of the cycle's growth and worth reconsidering against the band, which can now see it. RAISING OR LOWERING A ROW IS A DECISION, not a maintenance chore. Append here with the interpreter and the harness invocation, as above. -- 2026-09-26 #546 (its four conditions superseded by 2026-09-29 #561, below; its rows superseded by 2026-10-06 #617, and its link pin by #601's re-pointed shape on 2026-10-04, which this file records no entry for) -- the stages copy their state with `_state.copy_with` instead of `dataclasses.replace`, and every row moves DOWN. `dataclasses.replace` walks `fields()` and calls `__init__` on each copy, three frames on 3.11 and 3.12 and four from 3.13 where `copy_with` is one. One parse of the reference name makes 18 copies: six of the state, one each from tokenize, segment, classify, group, assign and post_rules, and twelve of single tokens, six each from classify and assign (`extract_delimited` returns the state unchanged when there is no delimiter, and `script_segment` returns early on ASCII input). That is the whole of the drop, 18 x 2 = 36 frames on 3.11 and 3.12 and 18 x 3 = 54 from 3.13 (recompute: wrap `copy_with` with a counter in the eight stage modules and parse the reference name). A field copy builds what `replace` builds for a dataclass that is decorated itself rather than inheriting the decoration, keeps the generated `__init__`, and has no `__post_init__` and no `init=False` field. `_copyable_fields` checks exactly those four, and `_COPY_FIELDS` runs it over `WorkToken`, `PendingAmbiguity` and `ParseState` at import, so a class that stops qualifying fails there and `copy_with` copies nothing else. `test_the_guard_refuses_a_class_a_field_copy_would_get_wrong` records, for each refused shape, what `replace` builds and what an unguarded copy would build instead. To mypy, `copy_with` is `from dataclasses import replace as copy_with`, which keeps the dataclass plugin's keyword and type checks at every call site; an assignment (`copy_with = dataclasses.replace`) would not, since the plugin keys on the callee's full name (measured: a misspelled field and a wrong-typed value both pass through the assignment and both fail through the import). Measured 2026-09-26 with `uv run python tools/perf/call_count.py --against e0f1a2f`, each row on its own interpreter, parse/facade: 3.11 406/443 → 370/407, 3.12 384/421 → 348/385, 3.13, 3.14 and 3.15 402/439 → 348/385. The rows drop by 40 and 58 rather than 36 and 54 because e0f1a2f already read 4 under every row, inside the band, and the new rows are set to what the harness reads now. `_LINK_BASELINE`'s 64-link clause reads 2587 → 2301 on 3.11. By stage (`--stages`, py3.11, ms per 1000 parses of the reference name): group 19.6 → 17.3, classify 13.4 → 9.8, assign 11.1 → 8.0, tokenize 8.1 → 6.6, post_rules 7.9 → 6.5, segment 2.6 → 1.7. BEHAVIOR IDENTICAL: the differential gate's report at all five baselines matches e0f1a2f's line for line apart from the path header. +- 2026-09-26 #546 (its four conditions superseded by 2026-09-29 #561, below; its rows superseded by 2026-10-06 #617, and its link pin by 2026-10-04 #601, below) -- the stages copy their state with `_state.copy_with` instead of `dataclasses.replace`, and every row moves DOWN. `dataclasses.replace` walks `fields()` and calls `__init__` on each copy, three frames on 3.11 and 3.12 and four from 3.13 where `copy_with` is one. One parse of the reference name makes 18 copies: six of the state, one each from tokenize, segment, classify, group, assign and post_rules, and twelve of single tokens, six each from classify and assign (`extract_delimited` returns the state unchanged when there is no delimiter, and `script_segment` returns early on ASCII input). That is the whole of the drop, 18 x 2 = 36 frames on 3.11 and 3.12 and 18 x 3 = 54 from 3.13 (recompute: wrap `copy_with` with a counter in the eight stage modules and parse the reference name). A field copy builds what `replace` builds for a dataclass that is decorated itself rather than inheriting the decoration, keeps the generated `__init__`, and has no `__post_init__` and no `init=False` field. `_copyable_fields` checks exactly those four, and `_COPY_FIELDS` runs it over `WorkToken`, `PendingAmbiguity` and `ParseState` at import, so a class that stops qualifying fails there and `copy_with` copies nothing else. `test_the_guard_refuses_a_class_a_field_copy_would_get_wrong` records, for each refused shape, what `replace` builds and what an unguarded copy would build instead. To mypy, `copy_with` is `from dataclasses import replace as copy_with`, which keeps the dataclass plugin's keyword and type checks at every call site; an assignment (`copy_with = dataclasses.replace`) would not, since the plugin keys on the callee's full name (measured: a misspelled field and a wrong-typed value both pass through the assignment and both fail through the import). Measured 2026-09-26 with `uv run python tools/perf/call_count.py --against e0f1a2f`, each row on its own interpreter, parse/facade: 3.11 406/443 → 370/407, 3.12 384/421 → 348/385, 3.13, 3.14 and 3.15 402/439 → 348/385. The rows drop by 40 and 58 rather than 36 and 54 because e0f1a2f already read 4 under every row, inside the band, and the new rows are set to what the harness reads now. `_LINK_BASELINE`'s 64-link clause reads 2587 → 2301 on 3.11. By stage (`--stages`, py3.11, ms per 1000 parses of the reference name): group 19.6 → 17.3, classify 13.4 → 9.8, assign 11.1 → 8.0, tokenize 8.1 → 6.6, post_rules 7.9 → 6.5, segment 2.6 → 1.7. BEHAVIOR IDENTICAL: the differential gate's report at all five baselines matches e0f1a2f's line for line apart from the path header. - 2026-09-29 #561 -- SUPERSEDED IN PART: the 2026-09-26 bullet above, whose four conditions ("decorated itself", "keeps the generated `__init__`", no `__post_init__`, no `init=False` field) were not sufficient. `replace` builds through `obj.__class__(...)`, and a field copy skips every hook that call runs; four shapes met all four conditions and still parted: a validating `__new__`, a validating `__setattr__` on a non-frozen dataclass and a validating metaclass `__call__`, each giving `ValueError` from `replace` where an unguarded field copy built `value=-1`, and an `InitVar` with no default, which `replace` demands (a `ValueError` through 3.12, a `TypeError` from 3.13) and a field copy never sees. #561's first tightening (80554266) found only the first three, and by swapping the class's OWN `__dataclass_params__` for an inherited lookup it also let through an `__init__` borrowed from another dataclass under that dataclass's name, which the 2026-09-26 guard had refused; review caught both. So the guard is now a list of conditions a class must MEET (`_state._copy_refusals`, one reason string per unmet condition): decorated as a dataclass itself, frozen, an `__init__` generated for it (compiled from `` and named `.__init__`) that takes exactly its `init` fields, the default metaclass, no `__new__` above `object`, no `__post_init__`, no `init=False` field. It is a TRIPWIRE for a realistic edit to the three pipeline classes and says so, not a proof against any class: a forged `__qualname__` on a borrowed `__init__` in a class that is also decorated itself, or a `__class__` property, still passes, and `copy_with` copies nothing but `WorkToken`, `PendingAmbiguity` and `ParseState`, which meet every condition on 3.11 to 3.15. `_UNGUARDED_EFFECT` records for each row what `replace` builds, what an unguarded copy builds, and which conditions refuse it; `test_every_guard_condition_alone_refuses_a_recorded_shape` asserts that each condition is the only refusal of some row, so dropping one fails a test rather than contradicting this paragraph (checked by removing the `__new__` condition in a scratch copy: two failures). Call counts unchanged, the guard running at import only. - 2026-10-10 #624 (its rows superseded by 2026-10-10 #625) -- every row moves DOWN by 4: `parse` 308 → 304 on 3.11 and 289 → 285 on 3.12-3.15, `facade` 345 → 341 and 326 → 322, each measured on its own interpreter with `tools/perf/call_count.py` in isolated copies of 8e6e524a (#631's merge) and of the change, the import asserted (8e6e524a read exactly the recorded rows). `--modules` on 3.11: `_pieces.py` 51 → 49 and `_group.py` 25 → 23 -- group's leading-position scan, a generator asking `is_leading_title` per piece, gave way to `leading_titles`, which it already called, and `chain_lead`, which asks `is_prefix_piece` only of the titles. - 2026-10-10 #625 -- every row moves UP by 3: `parse` 304 → 307 on 3.11 and 285 → 288 on 3.12-3.15, `facade` 341 → 344 and 322 → 325, each measured on its own interpreter with `tools/perf/call_count.py` in isolated copies of 505d6dec (#632's merge) and of the change, the import asserted (505d6dec read exactly the recorded rows). `--modules` on 3.11: `_pieces.py` 49 → 52, `_group.py` 23 → 23. The reference name holds a particle run (`de la`), so it pays the merge's call and one more `merge_pieces` and `joined_tags`, the chain then merging `de la` with `Vega` where it had merged the three words at once; the first cut asked the title run and the lead in the merge and again in the read, +11, and group now asks the title run once and hands both on. Every name of three or more words read at the trailing site pays the call, +1 (`John Q. Smith` 202 → 203 on 3.11, warm); a name with a particle run pays the 3 (`Juan de la Vega` 230 → 233). +- 2026-10-04 #601 -- THE LINK PIN IS RE-POINTED FROM THE MAIDEN CLAUSE TO THE NAME, 2,301 → 2,678 on py3.11. #601 retired the clause's link exception, so the clause shape the pin had measured reached `_between_name_words` 0 times; the same 64-link run written in the name (`Jane Puig i … Soler`) reaches it once per link, and a per-link regression moves it by 64 against a 2% band of 53.6. Recorded 2026-10-22 for #762 from commit 1037c87c and `tests/v2/test_benchmark.py`'s comment beside the pin, the move having landed with no entry here. - 2026-10-06 #617 (its rows and link pin superseded by 2026-10-07 #619) -- every row moves DOWN by 10: `parse` 374 → 364 on 3.11 and 353 → 343 on 3.12-3.15, `facade` 411 → 401 and 390 → 380, each measured on its own interpreter with `PYTHONPATH= pythonX.Y tools/perf/call_count.py` against 83f4e914 (every row had sat inside its band before, the 3.12-3.15 ones at +5 over a 348 baseline). `--modules` puts all ten in group and the piece layer (`_group.py` + `_pieces.py` 142 → 132, every other module unchanged): P3's connective joins moved out of `_group_segment` into `_pieces.join_connectives`, which asks `is_conj_piece`, `is_title_piece` and `is_prefix_piece` directly where the loops had asked through one-line closures over them, two frames per test becoming one. The link pin moves with it, 2,678 → 2,479 for the 64-link name on 3.11 (`_LINK_BASELINE`), and there for two reasons, counted per function in #617's review: the closures (`conj` 133 frames, `title` and `prefix` one each) and the single-letter test, which had joined each connective's text through a generator (65 frames, one per link, more than the pin's whole band), less the one `join_connectives` frame. A deliberate move, not drift: the change was a refactor (decisions/P3, 2026-10-06). The reference rows hold no comma; a comma part holding a connective now pays the shared loop (`John Smith, Mr. and Mrs.` 262 → 269 on 3.11, warm), and one holding none skips it and costs what it did (`Smith, John` 161), the review having found the first cut charging every family-comma parse two frames per word of the part. THEN 39 MORE on every row, in the branch's /simplify pass: group ran the rootname count (about five frames a piece, its only reader the carve-out) and the shared loop over every segment of three or more pieces, connective or not, and over the corpora and the case table under three orders 0 of the 3,153 group calls with no connective left free changed a piece. Group now skips both where its frozen-set walk, which already reads every piece's `conjunction` tag, finds no connective free, as `_comma._reading_pieces` does: `parse` 364 → 325 on 3.11 and 343 → 304 on 3.12-3.15, `facade` 401 → 362 and 380 → 341, `_group.py` + `_pieces.py` 132 → 93, measured on each interpreter as above; the link pin does not move, every link in it joining. Per name on 3.11, warm: the Vega name 436 → 386, `Smith, John Quincy Adams Bob` 326 → 297, `Smith, John` and `John Smith, Mr. and Mrs.` unchanged at 161 and 269. No output moved over the #617 grids (decisions/P3). - 2026-10-07 #619 (its rows superseded by 2026-10-07 #614; #619 added 2026-10-22, the head having named no issue of its own: commit 343e268c landed through #619's merge, adf6da88) -- every row moves DOWN by 8 more: `_group_segment`'s last three one-line predicate closures, `prefix` over `is_prefix_piece`, `suffix` over `is_suffix_piece` and `marker` over `_is_maiden_marker_piece`, are inlined as direct calls, finishing what #617 did for `title`, `conj` and `merge`. Each closure was a frame of its own ahead of the predicate's, and the chain's two inner scans ask `prefix` once per piece they pass, so the cost grows with particle pieces (frame-budget question 1 in AGENTS.md). Measured with `PYTHONSAFEPATH=1 PYTHONPATH= pythonX.Y tools/perf/call_count.py`, each row on its own interpreter, parse/facade against 0c54662b: 3.11 325/362 → 317/354, 3.12, 3.13, 3.14 and 3.15 304/341 → 296/333. `--modules` on 3.11 puts all eight in `_group.py` (34 → 26), every other module unchanged. The whole of the drop is the closures, counted per function on 0c54662b (a profile hook counting `call` events whose code is named `prefix`, `suffix` or `marker` in `_group.py`): the reference name asked `prefix` 7 times and `suffix` once, `Juan` + `de la Vega ` × 8 + `Smith` asked 40 and 9 and reads 937 → 888 on 3.11 (919 → 870 on 3.12-3.15), and the 64-link name asked `prefix` once, so `_LINK_BASELINE` moves 2,479 → 2,478 on 3.11 and gains rows for 3.12-3.15 at 2,459 (2,460 on 0c54662b), each measured on its own interpreter and the first the pin has had beyond 3.11; `marker` is reached only by P5's bound-join decline and the reference name never asks it. BEHAVIOR IDENTICAL: the full suite passes, and the differential gate's report at all five baselines matches 0c54662b's line for line apart from the path header. - 2026-10-08 #620 (its rows superseded by 2026-10-10 #624) -- no row moves: `parse` 308 on 3.11 and 289 on 3.12-3.15, `facade` 345 and 326, measured as the #614 bullet below on 4278693d (#621's merge) and on the change alike. The 3.12-3.15 pin read 290/327 until this date: #614's /simplify took one frame off those interpreters (289/326 on 4278693d, measured 2026-10-08) and the pin stayed, inside its band, so the #614 bullet's 290 is the pin's number, not that tree's. `--modules` on 3.11 unchanged. The merged copy and its per-word exclusion test are gone, and `_chain_units` asks `chain_run_end` per particle unit and `_weighed` per word behind one; the chain itself calls `chain_run_end` with the particle test inline where it had called `is_prefix_piece` per piece, which pays for the rest. decisions/S2, this date. diff --git a/docs/design/rules/R3-initials-honor-the-fold.md b/docs/design/rules/R3-initials-honor-the-fold.md index 5c41bb82..2a7d7e05 100644 --- a/docs/design/rules/R3-initials-honor-the-fold.md +++ b/docs/design/rules/R3-initials-honor-the-fold.md @@ -18,7 +18,7 @@ "Jon Dough and" → initials="J. D." "Juan de y" → initials="J. y." ``` -- **The first letter is the one the word's composed spelling begins with**, written in the form the writer used: a letter written as a base letter followed by its combining accent initials whole, and so does a syllable written as its separate parts. +- **The first letter is the one the word's composed spelling begins with**, written in the form the writer used: a letter written as a base letter followed by its combining accent initials whole, and so does a syllable written as its separate parts, wherever the two compose to one character; a mark that never composes with its letter (a nukta after ज, whose precomposed U+095B is excluded from composition) is left out of the initial, while the precomposed letter keeps it. ``` "émile zola" → initials="e\u0301. z." "マイケル ジャクソン" → initials="マ. シ\u3099." diff --git a/tests/v2/decisions_without_standing.json b/tests/v2/decisions_without_standing.json deleted file mode 100644 index e1a4eabd..00000000 --- a/tests/v2/decisions_without_standing.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "_comment": "decisions files allowed to lack a ## Standing block: those created before the rule, C1's block landing on 2026-10-19 (C12 and vocabulary-model were created that day, before it). A file leaves the list the day it gains a block, or the day the index lists it under Reference lists; the list must stay inside STANDING_EXEMPT_ORIGINAL in tests/v2/design_docs.py, so it never holds a key it did not start with and a new file never joins it (tests/v2/test_design_docs.py::test_a_decisions_file_off_the_list_opens_with_standing; #762 decides the rest).", - "exempt": [] -} diff --git a/tests/v2/design_docs.py b/tests/v2/design_docs.py index b3be9fe5..b3dfe3e9 100644 --- a/tests/v2/design_docs.py +++ b/tests/v2/design_docs.py @@ -14,7 +14,6 @@ """ from __future__ import annotations -import json import re import unicodedata from collections.abc import Set as AbstractSet @@ -157,8 +156,8 @@ def reused_id_findings(rule_ids: set[str], # A decisions file carries `## Standing`, an index of the decisions in -# force, ahead of its entries under `## Log` -- every file but those -# listed in decisions_without_standing.json (below). Each Standing +# force, ahead of its entries under `## Log` -- every file but the +# reference lists (below). Each Standing # bullet ends its claim with the entry that decided it, by date and # issue (`— 2026-10-06 #613`) or by date alone; a `Not standing:` line # names entries a later one answered. An entry is a log bullet opening @@ -426,48 +425,22 @@ def standing_findings(text: str) -> list[str]: return findings -#: The decisions files allowed to lack a Standing block: every file -#: created before the rule, C1's block landing on 2026-10-19 -#: (decisions/README.md, `## Standing`). It never holds a key it did not -#: hold when the rule landed. -STANDING_EXEMPT = Path(__file__).with_name("decisions_without_standing.json") -#: The list as recorded when the rule landed. Never edited: the list -#: must stay inside it, so a new file cannot join the list, alone or in -#: exchange for one that leaves it. -STANDING_EXEMPT_ORIGINAL = frozenset({ - '3-0-reevaluations', 'A1', 'A2', 'C10', 'C11', 'C12', 'C3', 'C4', - 'C5', 'C6', 'C7', 'C8', 'C9', 'D1', 'D2', 'H1', 'H2', 'H3', 'H4', - 'H5', 'H6', 'M1', 'M2', 'M3', 'M4', 'N1', 'N2', 'N3', 'O1', 'O2', - 'O4', 'O5', 'P1', 'P2', 'P3', 'P5', 'P6', 'P7', 'R1', 'R2', 'R3', - 'R4', 'R5', 'S10', 'S2', 'S3', 'S4', 'S5', 'S6', 'S7', 'S8', 'S9', - 'T1', 'T3', 'T4', 'W1', 'W2', 'W3', 'W4', 'api-churn-declines', - 'cjk-comma-demotion', 'cjk-full-stops', 'comma-suffix-arc', - 'comparison-surface', 'deviates-registry', 'differential-ledger', - 'generational-words', 'given-name-titles', 'indic-honorifics', - 'initials-repertoire', 'legacy-rule-numbers', 'ma-do', 'nfd-shapes', - 'normalization-fold', 'parse-cost', 'phd-merge', - 'removed-v1-surface', 'render-default', - 'review-agent-single-source', 'salutation-titles', - 'script-table-placement', 'suffix-acronym-collisions', - 'suffix-field-composition', 'two-input-invariants', - 'v1-xfail-triage', 'vocabulary-collisions', 'vocabulary-exclusions', - 'vocabulary-model'}) - - -def read_standing_exempt(path: Path = STANDING_EXEMPT) -> set[str]: - return set(json.loads(path.read_text(encoding="utf-8"))["exempt"]) - - # A reference list -- a file of Declined, Excluded, Open, removal or # mapping lines, each already saying what stands -- carries no Standing # block, which would copy it. The decisions index names them under this -# heading, and only a key the exempt list started with may be listed. A +# heading, and only the five in REFERENCE_LISTS may be listed: a new +# decisions file opens with ## Standing whatever it holds (#762). A # file there may not take a log's shapes: a dated entry, a bullet # opening with a tag and a date (`- (A) 2026-08-17 — …`, as # 3-0-reevaluations writes its log), or a dated title (a bundle, as # cjk-full-stops is). A log written in none of those shapes is caught # only in review. _REFERENCE_HEADING = "## Reference lists" +#: The reference lists, fixed when #762 finished giving every other +#: decisions file a Standing block (2026-10-22). +REFERENCE_LISTS = frozenset({ + 'api-churn-declines', 'given-name-titles', 'legacy-rule-numbers', + 'removed-v1-surface', 'vocabulary-exclusions'}) _LOG_SHAPED = re.compile( r"^- (?:\*\*)?(?:\([^)\n]{1,12}\) )?(?=\d{4}-\d\d)|\A# [^\n]*\d{4}-\d\d", re.M) @@ -493,17 +466,13 @@ def read_reference_lists(folder: Path = DECISIONS_DIR) -> set[str]: def standing_roster_findings( - files: dict[str, str], exempt: set[str], - original: frozenset[str] = STANDING_EXEMPT_ORIGINAL, - reference: AbstractSet[str] = frozenset()) -> list[str]: - """What breaks the Standing ratchet: a file off the exempt list with - no Standing block that is not a reference list, a listed file that - has gained one (take it off the list), a listed key no file carries - any more, a listed key the list did not hold when the rule landed, - and a heading that misspells `## Standing`; and of a reference list, - a Standing block, a log's shape (a dated or tagged-date bullet, a - dated title), a place on the exempt list, or a key the exempt list - did not hold when the rule landed.""" + files: dict[str, str], reference: AbstractSet[str], + allowed: AbstractSet[str] = REFERENCE_LISTS) -> list[str]: + """What breaks the Standing rule: a file with no Standing block that + is not a reference list, and a heading that misspells + `## Standing`; and of a reference list, a Standing block, a log's + shape (a dated or tagged-date bullet, a dated title), or a key + outside the fixed set.""" findings = [] for key in sorted(reference & files.keys()): text = files[key] @@ -514,30 +483,16 @@ def standing_roster_findings( findings.append(f"{key} is a reference list but is shaped " "like a log (a dated entry or title): a log " "opens with ## Standing") - if key in exempt: - findings.append(f"{key} is a reference list: take it off " - f"{STANDING_EXEMPT.name}") - for key in sorted(reference - original): - findings.append(f"{key} is under Reference lists but was not on " - f"{STANDING_EXEMPT.name} when the rule landed: a " - "new decisions file opens with ## Standing") + for key in sorted(reference - allowed): + findings.append(f"{key} is under Reference lists but is not one " + "of REFERENCE_LISTS: a decisions file opens with " + "## Standing") for key, text in sorted(files.items()): for line in misspelled_standing(text): findings.append(f"{key}: {line!r} is not the heading " "'## Standing'") - for key in sorted(exempt - original): - findings.append(f"{key} is listed in {STANDING_EXEMPT.name} but " - "was not on it when the rule landed: a new " - "decisions file opens with ## Standing instead") for key, text in sorted(files.items()): - has = has_standing(text) - if key in exempt and has: - findings.append(f"{key} has a Standing block: take it off " - f"{STANDING_EXEMPT.name}") - elif key not in exempt and key not in reference and not has: - findings.append(f"{key} has no Standing block: a new decisions " + if key not in reference and not has_standing(text): + findings.append(f"{key} has no Standing block: a decisions " "file opens with ## Standing and ## Log") - for key in sorted(exempt - files.keys()): - findings.append(f"{key} is listed in {STANDING_EXEMPT.name} " - "but no decisions file carries it") return findings diff --git a/tests/v2/rules_budget.json b/tests/v2/rules_budget.json index 786fb45f..087dae19 100644 --- a/tests/v2/rules_budget.json +++ b/tests/v2/rules_budget.json @@ -47,7 +47,7 @@ "P9": {"words": 405, "then_words": 265, "because_sentences": 1, "except": 2, "precedence": 0, "accepted": 0, "accepted_words": 0}, "R1": {"words": 164, "then_words": 133, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 1, "accepted_words": 140}, "R2": {"words": 143, "then_words": 19, "because_sentences": 1, "except": 1, "precedence": 0, "accepted": 2, "accepted_words": 85}, - "R3": {"words": 445, "then_words": 369, "because_sentences": 2, "except": 1, "precedence": 0, "accepted": 3, "accepted_words": 409}, + "R3": {"words": 483, "then_words": 407, "because_sentences": 2, "except": 1, "precedence": 0, "accepted": 3, "accepted_words": 409}, "R4": {"words": 913, "then_words": 651, "because_sentences": 1, "except": 4, "precedence": 0, "accepted": 3, "accepted_words": 552}, "R5": {"words": 250, "then_words": 154, "because_sentences": 2, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0}, "S1": {"words": 274, "then_words": 26, "because_sentences": 1, "except": 2, "precedence": 2, "accepted": 0, "accepted_words": 0}, diff --git a/tests/v2/test_benchmark.py b/tests/v2/test_benchmark.py index 05e2b204..313f3c8c 100644 --- a/tests/v2/test_benchmark.py +++ b/tests/v2/test_benchmark.py @@ -88,7 +88,8 @@ #: re-asked by the chain and the reserve and read again by assign #: (decisions/parse-cost). The 3.12-3.15 rows read 289/326 from #: #614's /simplify on and were corrected in #620, which moved no row -#: (decisions/parse-cost). +#: (decisions/parse-cost). Then down by 4 on every row in #624 and up +#: by 3 in #625, to the figures below (decisions/parse-cost). _CALL_BASELINE = { (3, 11): {"parse": 307, "facade": 344}, (3, 12): {"parse": 288, "facade": 325}, diff --git a/tests/v2/test_design_docs.py b/tests/v2/test_design_docs.py index 72e911ee..8a467461 100644 --- a/tests/v2/test_design_docs.py +++ b/tests/v2/test_design_docs.py @@ -8,9 +8,9 @@ from tests.v2.design_docs import ( DECISIONS_DIR, MENTION_RE, RULES_DIR, read_decision_files, read_index, - STANDING_EXEMPT_ORIGINAL, _POINTER_RE, _clean, _entries, _hits, + REFERENCE_LISTS, _POINTER_RE, _clean, _entries, _hits, has_standing, read_reference_lists, read_rules_text, - read_standing_exempt, reference_lists, resolve, retired_ids, + reference_lists, resolve, retired_ids, reused_id_findings, rule_file_ids, slugify, standing_findings, standing_roster_findings, stem_for) @@ -390,40 +390,24 @@ def test_every_standing_line_points_at_a_live_entry() -> None: assert standing_findings(files[key]) == [], key -_ORIGINAL = frozenset({"P2", "A1"}) - - -@pytest.mark.parametrize("files, exempt, finding", [ - ({"C1": "## Standing", "P2": "log", "A1": "log"}, {"P2", "A1"}, None), +@pytest.mark.parametrize("files, finding", [ + ({"C1": "## Standing", "L": "- Declined: x."}, None), ({"C1": "## Standing", "C14": "- 2026-10-20 — the `## Standing` rule.", - "P2": "log", "A1": "log"}, {"P2", "A1"}, "C14 has no Standing"), - ({"C1": "## Standing", "P2": "log", "A1": "## Standing"}, {"P2"}, - None), - ({"C1": "## Standing", "C14": "log", "P2": "log", "A1": "log"}, - {"P2", "A1"}, "C14 has no Standing"), - ({"C1": "## Standing", "P2": "## Standing", "A1": "log"}, {"P2", "A1"}, - "P2 has a Standing block: take it off"), - ({"C1": "## Standing", "A1": "log"}, {"P2", "A1"}, - "P2 is listed in decisions_without_standing.json but no decisions"), - ({"P2": "log", "A1": "log", "Z9": "log"}, {"P2", "A1", "Z9"}, - "was not on it when the rule landed"), - ({"C1": "## Standing", "P2": "log", "A1": "## Standing \n- a"}, - {"P2", "A1"}, "'## Standing ' is not the heading"), - ({"C1": "## Standing", "P2": "log", "A1": "### Standing\n- a"}, - {"P2", "A1"}, "'### Standing' is not the heading"), - ({"C1": "## Standing", "P2": "log", "A1": " ## Standing\n- a"}, - {"P2", "A1"}, "' ## Standing' is not the heading"), - ({"P2": "log", "A1": "## Standing", "Z9": "log"}, {"P2", "Z9"}, - "was not on it when the rule landed"), -], ids=["clean", "prose-mention-is-no-block", "listed-file-left", - "new-file-without", - "listed-file-gained-one", "listed-file-gone", "new-file-listed", + "L": "- Declined: x."}, "C14 has no Standing"), + ({"C1": "## Standing", "C14": "log", "L": "- Declined: x."}, + "C14 has no Standing"), + ({"C1": "## Standing", "L": "## Standing \n- a"}, + "'## Standing ' is not the heading"), + ({"C1": "## Standing", "L": "### Standing\n- a"}, + "'### Standing' is not the heading"), + ({"C1": "## Standing", "L": " ## Standing\n- a"}, + "' ## Standing' is not the heading"), +], ids=["clean", "prose-mention-is-no-block", "file-without", "misspelled-heading-trailing-space", "misspelled-heading-level", - "misspelled-heading-indented", - "new-file-swapped-in"]) + "misspelled-heading-indented"]) def test_standing_roster_findings_name_each_break( - files: dict[str, str], exempt: set[str], finding: str | None) -> None: - found = standing_roster_findings(files, exempt, _ORIGINAL) + files: dict[str, str], finding: str | None) -> None: + found = standing_roster_findings(files, {"L"}, frozenset({"L"})) if finding is None: assert found == [] else: @@ -448,58 +432,55 @@ def test_reference_lists_are_the_links_under_their_heading() -> None: assert reference_lists("## Named topics\n\n- [a — x](a.md)\n") == set() -@pytest.mark.parametrize("files, exempt, finding", [ - ({"C1": "## Standing", "L": "- Declined: x."}, set(), None), - ({"C1": "## Standing", "L": "- Declined: x.\n- 2026-10-01 — y."}, set(), +@pytest.mark.parametrize("files, finding", [ + ({"C1": "## Standing", "L": "- Declined: x."}, None), + ({"C1": "## Standing", "L": "- Declined: x.\n- 2026-10-01 — y."}, "L is a reference list but is shaped like a log"), ({"C1": "## Standing", "L": "- Declined: x.\n- (A) 2026-08-17 — parked."}, - set(), "L is a reference list but is shaped like a log"), - ({"C1": "## Standing", "L": "- **2026-10-01 #5 — y.**"}, set(), + "L is a reference list but is shaped like a log"), + ({"C1": "## Standing", "L": "- **2026-10-01 #5 — y.**"}, "L is a reference list but is shaped like a log"), ({"C1": "## Standing", "L": "# L — a bundle (2026-09-10, #322)\n\n- **NFKC declined.**"}, - set(), "L is a reference list but is shaped like a log"), + "L is a reference list but is shaped like a log"), ({"C1": "## Standing", "L": "# L — x\n\n- abd — ADDED 2026-08-18, see decisions/P5.\n" "- (Derek's call) removed 2026-08-26.\n"}, - set(), None), - ({"C1": "## Standing", "L": "## Standing\n- x"}, set(), + None), + ({"C1": "## Standing", "L": "## Standing\n- x"}, "L is a reference list and carries ## Standing"), - ({"C1": "## Standing", "L": "- Declined: x."}, {"L"}, - "L is a reference list: take it off"), - ({"C1": "## Standing", "N": "- Declined: x."}, set(), - "N is under Reference lists but was not on"), + ({"C1": "## Standing", "N": "- Declined: x."}, + "N is under Reference lists but is not one"), ], ids=["list", "list-with-entry", "list-with-tagged-date", "list-with-bold-entry", "list-with-dated-title", "list-with-dates-inside-lines", "list-with-standing", - "list-also-exempt", "list-new-since-the-rule"]) + "list-new-since-the-rule"]) def test_a_reference_list_carries_no_standing_and_no_log( - files: dict[str, str], exempt: set[str], finding: str | None) -> None: + files: dict[str, str], finding: str | None) -> None: listed = {"N"} if "N" in files else {"L"} - found = standing_roster_findings(files, exempt, frozenset({"L"}), - reference=listed) + found = standing_roster_findings(files, listed, frozenset({"L"})) if finding is None: assert found == [] else: assert len(found) == 1 and finding in found[0], found # Off the heading, the same file is a log with no block. if finding is None: - assert standing_roster_findings(files, exempt, frozenset({"L"})) == [ - "L has no Standing block: a new decisions file opens with " + assert standing_roster_findings(files, set(), frozenset({"L"})) == [ + "L has no Standing block: a decisions file opens with " "## Standing and ## Log"] -def test_a_decisions_file_off_the_list_opens_with_standing() -> None: +def test_every_decisions_file_opens_with_standing_or_is_a_reference_list( + ) -> None: assert standing_roster_findings(read_decision_files(), - read_standing_exempt(), - reference=read_reference_lists()) == [] + read_reference_lists()) == [] -def test_the_original_exempt_set_is_never_edited() -> None: - # The list the rule landed with: 88 decisions files created before - # C1's block. A key added here would let a new file skip its block, - # so the size is asserted rather than trusted to the comment. - assert len(STANDING_EXEMPT_ORIGINAL) == 88 +def test_the_reference_lists_are_a_fixed_five() -> None: + # Fixed when #762 finished (2026-10-22): a key added here would let a + # new file skip its block, so the size is asserted rather than + # trusted to the comment. + assert len(REFERENCE_LISTS) == 5 _RETIRED_ENTRY = ("# X9 — a retired rule\n\n## Standing\n\n" From ee6029ebb59ca6833f38f9671c8fb6aef1affa0d Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Thu, 22 Oct 2026 14:28:18 -0700 Subject: [PATCH 3/3] Scope the same-day dating rule to undated bullets; mark #601's pin superseded; give R1's correction both policies Review of da3e8551: the split README sentence let a same-day bullet whose text gives a date keep the bundle's date, contradicting the cjk-full-stops line; parse-cost's #601 entry presented 2,678 as current though #617 moved it; R1's correction gave the default policy's before-reading only. Co-Authored-By: Claude Opus 5.5 --- .../design/decisions/R1-fields-are-views-of-the-parsed-words.md | 2 +- docs/design/decisions/README.md | 2 +- docs/design/decisions/parse-cost.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/design/decisions/R1-fields-are-views-of-the-parsed-words.md b/docs/design/decisions/R1-fields-are-views-of-the-parsed-words.md index 8dac9d86..5c8d6c73 100644 --- a/docs/design/decisions/R1-fields-are-views-of-the-parsed-words.md +++ b/docs/design/decisions/R1-fields-are-views-of-the-parsed-words.md @@ -39,5 +39,5 @@ Open: none in this file. DECLINED, all four with the evidence: marking the boundary at the core-drop site (#437's MARK-DONT-STRIP shape) — `dropped` already holds the fact with its span, and a second recording of it is the duplication that mechanism exists to prevent, one level up; making the `"joined"` tag role-aware — within a piece it is role-blind and correct for every role, `Smith, Ph. D. Smith` giving `first_list == ['Ph. D.']`, and only the between-piece half was ever a suffix concept; a render-time span scan instead of the recorded tag — `_facade.__setstate__` and `ParsedName.replace()` synthesise span-less tokens, so an unpickled name has nothing to scan and the tag IS the entry structure a pickle carries; and fixing the no-comma path inside group's block as a third branch beside `tail` and `reading`, which is the shape #429 took. - 2026-09-06 #511 — a suffix value handed to `revise()` derives its entries from its own commas, by the rule a whole name uses. `Parser.revise` sub-parsed each value and forced every harvested token to the named role AFTER the sub-parse had run R1's entry pass, which keys on Role.SUFFIX; a bare 'MD PhD' reads there as a title and a family name, so the pass joined nothing and the field rendered 'MD, PhD'. The fix runs the same pass again: `revise` now sub-parses to a ParseState, forces the role on every non-dropped token, and calls `suffix_entries` — the pass, lifted unedited out of post_rules' tail into a function the two callers share (mechanisms.md#ONE-PREDICATE-PER-QUESTION) — over the forced state, then assembles and harvests as before. So a comma in the value parts two credentials and a space joins them. TWO SPELLINGS of the lifted pass, and the reason is the call budget: a first draft had post_rules call the state-in/state-out wrapper, and the second ParseState build cost three more calls per parse against the band tests/v2/test_benchmark.py holds (py3.11, 2026-09-06, tools/perf/call_count.py: 450 calls/name before the move, 451 with the in-place worker `_mark_suffix_entries` that post_rules now calls, 454 with the draft; the facade band tops at 455.9), so the worker writes in place as every other post rule does and the wrapper exists for the one caller with no token list of its own. NO `_run` HELPER for the same reason: a draft routed `parse()` through a private state builder shared with `revise`, and that cost one frame per parse on the hot path (py3.11, 2026-09-06: 415 calls/name against 414 without it, in a 402-418 band), so the four-line construction is spelled twice and `parse()` is untouched. MEASURED 2026-09-06 over the 1117 distinct corpus names, revising each of the 368 with a non-empty suffix by its own suffix: 38 suffixes moved before and 1 after, and the differential gate was byte-identical at all four baselines (recipe in the #511 issue body). THE ONE LEFT is '김민준씨, J.씨', and it is not entry structure: the whole-name parse keeps 'J.씨' one glued suffix token, suffix '씨, J.씨', while the sub-parse of the bare value '씨, J.씨' peels the honorific off the initial, so the revised field renders '씨, J. 씨' — right entries, the spurious comma of before ('씨, J., 씨') gone, one word read differently by the value's own parse than by the whole name's. That is the "classified ON ITS OWN" limit `revise`'s docstring has always recorded, and CJK honorific peeling is a W-rule question this change does not move; pinned in `test_revise_reads_a_glued_honorific_on_its_own`. SUPERSEDES the phd-merge acceptance of 'Ph., D.' on this path (its bullet says how). STALE TAGS, tried and backed out: a draft cleared the sub-parse's own "joined" with the forcing and re-derived it, because the pass only ADDS the tag and `revise(n, family="Jones MD PhD")` carries the sub-parse's between-piece suffix mark on 'PhD' onto a FAMILY token. Measured 2026-09-06 against `330ee55`, the clear also destroyed every WITHIN-piece mark on a non-suffix value — 'D.' of `revise(n, family="John Ph. D. Smith")` lost the merge mark the #436 bullet's DECLINED list calls role-blind and correct for every role — while on a ParsedName only the suffix string view reads "joined" (`_text_for`'s suffix_join gate) — the facade's `_list_for` heals it for every role, but no path puts a revised name into a HumanName, the v1 setters going through `replace()`, so wiring those setters onto `revise` is the change that would show the stale mark, as `last_list == ['Jones', 'MD PhD']` — `initials()` and every field string being identical at both trees for every shape measured. So the tags are kept minus FOLDED_TAG as before; the between-piece mark on a forced non-suffix role is a tag-only oddity that predates this change and stays, and for a suffix value nothing depends on the sub-parse's marks, every pair it joined sharing a bucket with no parting token and the pass setting it again; pinned in `test_revise_keeps_the_sub_parses_within_piece_mark`. Dropped tokens keep their role and tags through the forcing, because the pass filters them by index and assemble omits them. ONE MORE LIMIT, pinned in `test_revise_leaves_a_policy_delimiter_unparted_without_a_tail_segment`, and it is the "classified ON ITS OWN" limit again rather than a rule of revise's: a delimiter the policy names through `extra_suffix_delimiters` is dropped, and so parts entries, only on a segment after a comma that the reading of the words makes a tail, and a value with no comma of its own has none — under `Policy(extra_suffix_delimiters=frozenset({" - "}))` the whole name 'Doe, John, MD PhD - FACS' renders 'MD PhD, FACS' while `revise(n, suffix="MD PhD - FACS")` renders 'MD PhD - FACS', the dash surviving as a token and the forced role making it a suffix word; measured 2026-09-06, a value whose own words read with a tail segment does part ('John Doe, MD - FACS' revises to 'John Doe, MD, FACS') and one after a suffix comma does not ('MD, PhD - FACS' stays), which is how the value's words read as a name deciding it. The round-trip is unaffected, the whole-name view having already rendered that boundary as a comma. DECLINED: a list-valued `revise(p, suffix=["MD PhD", "FACS"])`, which widens the API and leaves the string path where it was; documenting the limit with pins alone, the docstring already recording it and #511's measurement showing it reachable on every space-joined run #436 produced; and a text-only read of the delimiter cores inside `revise`, a second rule for a case no round-trip reaches. Out of scope and left as is: `ParsedName.replace(suffix="MD, PhD")` renders 'MD,, PhD', `replace` whitespace-splitting by contract and the facade setters riding on it for v1 parity. - 2026-10-18 #728 — MOVED FROM decisions/C1. These entries decide that one comma segment is one suffix entry, which is R1's entry pass rather than the comma decision, and were filed under decisions/C1 until #728; they moved unchanged but for the pointers the move broke, the measurement paragraphs, each cut to one line naming its population, counts, baseline and PR, and the superseded passages, each cut to its decision and what replaced it. -- 2026-10-19 #760 — MOVED FROM the entry pass's code comment (`_post_rules._mark_suffix_entries`), the facts the entries above did not hold. ORDER: the pass runs after assign because it keys on Role.SUFFIX; the same span rule run role-blind would join the A and B of `John A B Smith` into one middle_list element, pinned by `test_the_pass_runs_after_roles_are_settled` in tests/v2/pipeline/test_post_rules.py. Within post_rules it runs last by convention, not by necessity: every retag there targets a name role (`_retag` is called with FAMILY, GIVEN, MIDDLE and `_name_positions`' return, which is those three, and the `role=Role.FAMILY` replaces of P6's no-comma attachment and O3's fold are FAMILY too), so no rule moves a token into or out of SUFFIX or {TITLE, NICKNAME, MAIDEN} and the predicate reads the same answer wherever in the stage it stands (measured 2026-09-06 by reading the stage's retag targets). THE UNGATED CORE READ: group drops a core only on a `tail` segment, while the pass reads `delimiter_cores` whole and asks by TEXT alone, so a dropped token whose text the policy names parts the run whatever dropped it. The gate is not needed: a core group dropped was on a tail segment by construction, and the one case the ungated read adds is a maiden marker the policy also lists — under `Policy(extra_suffix_delimiters=frozenset({" nee "}))`, `John Doe, MD nee Jones PhD` renders 'MD, PhD' where the default policy renders 'MD PhD' (measured 2026-09-06), the writer's configuration having named that text a separator. That example no longer reaches the case: re-measured 2026-10-19 while moving it here, `John Doe, MD nee Jones PhD` reads title 'MD', given 'nee', middle 'Jones', family 'John Doe' and suffix 'PhD' under both policies, the comma now read as a family comma, so the marker is never dropped and the pass sees one post-nominal; `Smith, MD nee Jones PhD`, the bare-marker example of the #436/#437 entry above, reads the same way. (Corrected 2026-10-22, #762: both moved with #601, measured either side of 1037c87c, where before it they read maiden 'Jones' with suffix 'MD PhD'; what moved is the marker, which no longer takes a clause in that part, and for `Smith, …` the comma was a family comma before as after.) The witness today is the bracketed clause: `Smith, MD (nee Jones) PhD` renders 'MD PhD' under the default policy and 'MD, PhD' under that one, extract having dropped the marker as Role.MAIDEN and group never dropping it, so the ungated text read is what parts the run (re-measured 2026-10-19 in #760's review). The same input is the filter's example below. THE DROPPED-INDEX FILTER is belt-and-braces: cores leave `pieces` before assign and get no role, and the one dropped token that arrives roled is the MAIDEN one tokenize roles from an extracted clause (`Smith, MD (nee Jones) PhD` drops index 2, the marker, as Role.MAIDEN); the filter keeps the suffix list and the parting scan agreeing about what a dropped index is. +- 2026-10-19 #760 — MOVED FROM the entry pass's code comment (`_post_rules._mark_suffix_entries`), the facts the entries above did not hold. ORDER: the pass runs after assign because it keys on Role.SUFFIX; the same span rule run role-blind would join the A and B of `John A B Smith` into one middle_list element, pinned by `test_the_pass_runs_after_roles_are_settled` in tests/v2/pipeline/test_post_rules.py. Within post_rules it runs last by convention, not by necessity: every retag there targets a name role (`_retag` is called with FAMILY, GIVEN, MIDDLE and `_name_positions`' return, which is those three, and the `role=Role.FAMILY` replaces of P6's no-comma attachment and O3's fold are FAMILY too), so no rule moves a token into or out of SUFFIX or {TITLE, NICKNAME, MAIDEN} and the predicate reads the same answer wherever in the stage it stands (measured 2026-09-06 by reading the stage's retag targets). THE UNGATED CORE READ: group drops a core only on a `tail` segment, while the pass reads `delimiter_cores` whole and asks by TEXT alone, so a dropped token whose text the policy names parts the run whatever dropped it. The gate is not needed: a core group dropped was on a tail segment by construction, and the one case the ungated read adds is a maiden marker the policy also lists — under `Policy(extra_suffix_delimiters=frozenset({" nee "}))`, `John Doe, MD nee Jones PhD` renders 'MD, PhD' where the default policy renders 'MD PhD' (measured 2026-09-06), the writer's configuration having named that text a separator. That example no longer reaches the case: re-measured 2026-10-19 while moving it here, `John Doe, MD nee Jones PhD` reads title 'MD', given 'nee', middle 'Jones', family 'John Doe' and suffix 'PhD' under both policies, the comma now read as a family comma, so the marker is never dropped and the pass sees one post-nominal; `Smith, MD nee Jones PhD`, the bare-marker example of the #436/#437 entry above, reads the same way. (Corrected 2026-10-22, #762: both moved with #601, measured either side of 1037c87c, where before it they read maiden 'Jones' with suffix 'MD PhD' under the default policy and 'MD, PhD' under the `" nee "` one, the contrast this paragraph was written to show; what moved is the marker, which no longer takes a clause in that part, and for `Smith, …` the comma was a family comma before as after.) The witness today is the bracketed clause: `Smith, MD (nee Jones) PhD` renders 'MD PhD' under the default policy and 'MD, PhD' under that one, extract having dropped the marker as Role.MAIDEN and group never dropping it, so the ungated text read is what parts the run (re-measured 2026-10-19 in #760's review). The same input is the filter's example below. THE DROPPED-INDEX FILTER is belt-and-braces: cores leave `pieces` before assign and get no role, and the one dropped token that arrives roled is the MAIDEN one tokenize roles from an extracted clause (`Smith, MD (nee Jones) PhD` drops index 2, the marker, as Role.MAIDEN); the filter keeps the suffix list and the parting scan agreeing about what a dropped index is. - 2026-10-20 #761 — MOVED FROM rules/R1's revise() Accepted clause when R1 moved to the skeleton format, the measurement it carried: the #511 bullet counted 368 of 1117 distinct corpus names with a suffix on 2026-09-06; re-measured 2026-09-10 with that bundle's rows in the corpora, 372 of 1156 distinct names, and the same single failure, '김민준씨, J.씨'. diff --git a/docs/design/decisions/README.md b/docs/design/decisions/README.md index 6d8cbab3..a7933a6e 100644 --- a/docs/design/decisions/README.md +++ b/docs/design/decisions/README.md @@ -11,7 +11,7 @@ Entry conventions: 2. **A factual error is corrected in place**, with a note saying what was wrong — a date, an attribution, a measurement or a pointer. decisions/C1's 2026-07-12/13 entry, whose parenthetical moved a misdated credit, is the shape. 3. **Compaction, on a stated trigger.** A superseded entry is cut to a tombstone: its date, issue, verdict line and `SUPERSEDED by`, plus whatever it recorded that the superseding entry did not replace — a gap it found standing before its change, or an option it declined that the reversal does not answer, stays beneath the tombstone (decisions/T4's #731); a superseded passage inside a live entry is cut to its decision and what replaced it. A measurement paragraph (what moves, cost, blast radius) is cut to one line naming its population, counts, baseline and PR once its change has shipped in a release. The full text stays in git and in the PR the entry cites. #728 is the worked compaction. A harvested entry (below) keeps its framing line through compaction, a tombstone included, and its measured-vs-remembered markings wherever its measurements survive. A trap #759's review found: a one-line replacement carries each count with the classification the old text gave that count and no other (decisions/S2's #614 line had given a whole grid the classification measured on one slice of it). 4. **A per-word fact that moves into the vocabulary's TOML is REPLACED by a query that finds it** (`uv run python tools/vocab/query.py --issue 661`, `--excluded generation`), not left beside a dated "moved" note, which would keep two copies to drift. Only the per-word sentences go; the decision, its criterion and its measurement stay as written. Sanctioned since 2026-10-18 (decisions/vocabulary-model). -- **`## Standing`**, at the head of a file: the one part edited freely. One line per decision in force, in the present tense, ending with the entry that decided it by its exact date and the first issue its head names (`— 2026-10-06 #613`), or by its date alone where the head names none; where several entries share that date and issue, the pointer goes on with the words its entry's head continues with (`— 2026-09-14 (Derek)`, `— 2026-10-18 #718 review`), the longest such continuation winning and a date inside those words being part of them; a pointer with no words names the entry whose head ends there, and one whose words continue no head there fails, since nothing tells a typo from prose, so prose after such a pointer goes after a `.` or `;` (the words end there, or where the next pointer's date begins); where two heads are the same words, the pointer goes on with the words its entry's text opens with (`— 2026-09-13 #383/#479, AMENDS`), a key rather than prose, and so it does where another head of that date and issue continues past those words (decisions/M3's `— 2026-08-26 #335, the MECHANISM`, beside its `#335 (M3, …)` head); and a pointer still naming more than one entry fails as ambiguous (a parenthesized supersession note in a head is not read), plus a `Not standing:` line for entries whose plan was carried out or whose schedule events overtook, its pointers read and failing as a Standing line's do except that they may name a superseded entry — a decision reversed by a later DECISION is marked under 1 instead and an `Open:` line for each open question. It indexes the log and is never a decision itself: a Standing line with no entry behind it, or only a superseded one, fails `tests/v2/test_design_docs.py::test_every_standing_line_points_at_a_live_entry`, and changing a decision is still a new entry, which updates Standing in the same diff. The entries follow under `## Log`. An entry is a log bullet opening with its date, bare or in bold (`- **2026-10-14 #679 — …**`); a file's title founds no entry, so a bundle recorded as undated bullets under a dated title gains, when its block is written, one dated line opening its Log and naming the bundle, its verdict the title's words (`- 2026-09-10 #322/#323 — WHICH CHARACTERS ARE A PERIOD ON CJK TEXT: the bullets below.`). It dates what the title already dates, so it adds no decision and edits no entry. A file with no dated title dates its bundle line by the date its preamble gives the bullets. A bullet decided on another day gains a dated line of its own beside the bundle line, naming the bullet by the words it opens with, since a pointer reads only a bullet that opens with its date (decisions/3-0-reevaluations, decisions/comma-suffix-arc, decisions/cjk-full-stops). The date is the decision's: the one the bullet's own text gives it (`- (A) 2026-08-17 — …`, `Amended 2026-08-23`; a date given for something else, such as when code landed, is not one), or, where it gives none, the author date of the commit that added the bullet. Find that commit with `git log -S --reverse` over `docs/design/`, not over the file, which every bullet entered on 2026-10-17 when #713 split decisions.md, and search a phrase short enough to have fit one line before the 2026-08-16 unwrap (5e2ae60b), a first hit there meaning the phrase crossed a wrapped line. A bullet written the day its bundle was keeps the bundle line's date, even where that date is older (decisions/comma-suffix-arc's Declined bullets). Where a later decision reversed part of the bundle, its head says which, under edit kind 1. That note is for the reader: a pointer never reads a parenthesized note containing "superseded" (edit kind 1's wording), so a Standing line resting on a bundle is checked by hand against what the bundle's later entries and other files reversed, as for any entry superseded in part. Every decisions file opens with one except the reference lists (below), a new file's cheapest moment to write it being its first entry; the files that predated the rule (C1's block landed 2026-10-19) were given theirs under #762, finished 2026-10-22, and the exempt list that held them while it ran is retired. `test_every_decisions_file_opens_with_standing_or_is_a_reference_list` fails on a file with no block that is not a reference list (the heading on a line of its own; the words quoted in prose do not count). The block stands ahead of every `## Log` heading; a `## Log` before it is a finding, and so is a heading that misspells `## Standing` (a trailing space, a different level), which would otherwise switch the pointer checks off. A decision of another rule's that this log is the record of stays a bullet naming its rule (decisions/C13's line for rules/C2, decisions/P9's for rules/S5); one recorded under another key is an `Elsewhere:` line pointing at that entry, not a bullet. A file of Declined, Excluded, Open, removal or mapping lines is a reference list instead, indexed under the Reference lists heading at the end of the index: it carries no block. A listed file carrying a block or taking a log's shape (a dated entry, a bullet opening with a tag and a date as `- (A) 2026-08-17 — …`, or a dated title) is a finding, as is a listed key outside the five fixed in `tests/v2/design_docs.py`'s `REFERENCE_LISTS` (2026-10-21, #762; fixed 2026-10-22). Whether a file in none of those shapes is a list or a log is a judgment the check cannot make, made in review. 3-0-reevaluations is not one: other files amend its lines without its saying so (decisions/cjk-comma-demotion scopes its FAMILY_COMMA line), the shape a Standing block's `Not standing:` and `Elsewhere:` lines are for. Five files split out on 2026-10-19 (C13, C14, P8, P9, T5) showed the rule was needed: none had a block, though C1's had landed the same day. **Before writing a line, look for what reversed it elsewhere.** A decision can be overturned by an entry in another file, which the log it stands in never mentions; #776 found two lines wrong that way, one while writing and one in review (P9's #578 note, reversed by decisions/S8's #602; P8's subset, widened by decisions/P3's #397), and #762's first batch (2026-10-21) found five entries whose heads did not say what had replaced part of them (decisions/S6's accepted `abdul Smith Jr Ma`, reversed by decisions/S7's #544; decisions/S4's `Doe, Jane nee Smith MA`, by decisions/M2's #601; decisions/H3's `Dr Smith V`, by decisions/S10's #718, caught only in that batch's PR review; decisions/P5's view comparison and decisions/S10's one-case `v`, by their own files' #614 and #721). So grep the whole folder for each issue and example a line cites, parse each example on the tree, and where a later decision reversed part of an entry mark that entry's head under 1, so the log says what Standing says. +- **`## Standing`**, at the head of a file: the one part edited freely. One line per decision in force, in the present tense, ending with the entry that decided it by its exact date and the first issue its head names (`— 2026-10-06 #613`), or by its date alone where the head names none; where several entries share that date and issue, the pointer goes on with the words its entry's head continues with (`— 2026-09-14 (Derek)`, `— 2026-10-18 #718 review`), the longest such continuation winning and a date inside those words being part of them; a pointer with no words names the entry whose head ends there, and one whose words continue no head there fails, since nothing tells a typo from prose, so prose after such a pointer goes after a `.` or `;` (the words end there, or where the next pointer's date begins); where two heads are the same words, the pointer goes on with the words its entry's text opens with (`— 2026-09-13 #383/#479, AMENDS`), a key rather than prose, and so it does where another head of that date and issue continues past those words (decisions/M3's `— 2026-08-26 #335, the MECHANISM`, beside its `#335 (M3, …)` head); and a pointer still naming more than one entry fails as ambiguous (a parenthesized supersession note in a head is not read), plus a `Not standing:` line for entries whose plan was carried out or whose schedule events overtook, its pointers read and failing as a Standing line's do except that they may name a superseded entry — a decision reversed by a later DECISION is marked under 1 instead and an `Open:` line for each open question. It indexes the log and is never a decision itself: a Standing line with no entry behind it, or only a superseded one, fails `tests/v2/test_design_docs.py::test_every_standing_line_points_at_a_live_entry`, and changing a decision is still a new entry, which updates Standing in the same diff. The entries follow under `## Log`. An entry is a log bullet opening with its date, bare or in bold (`- **2026-10-14 #679 — …**`); a file's title founds no entry, so a bundle recorded as undated bullets under a dated title gains, when its block is written, one dated line opening its Log and naming the bundle, its verdict the title's words (`- 2026-09-10 #322/#323 — WHICH CHARACTERS ARE A PERIOD ON CJK TEXT: the bullets below.`). It dates what the title already dates, so it adds no decision and edits no entry. A file with no dated title dates its bundle line by the date its preamble gives the bullets. A bullet decided on another day gains a dated line of its own beside the bundle line, naming the bullet by the words it opens with, since a pointer reads only a bullet that opens with its date (decisions/3-0-reevaluations, decisions/comma-suffix-arc, decisions/cjk-full-stops). The date is the decision's: the one the bullet's own text gives it (`- (A) 2026-08-17 — …`, `Amended 2026-08-23`; a date given for something else, such as when code landed, is not one), or, where it gives none, the author date of the commit that added the bullet. Find that commit with `git log -S --reverse` over `docs/design/`, not over the file, which every bullet entered on 2026-10-17 when #713 split decisions.md, and search a phrase short enough to have fit one line before the 2026-08-16 unwrap (5e2ae60b), a first hit there meaning the phrase crossed a wrapped line. A bullet whose text gives no date and that was written the day its bundle was keeps the bundle line's date, even where that date is older (decisions/comma-suffix-arc's Declined bullets). Where a later decision reversed part of the bundle, its head says which, under edit kind 1. That note is for the reader: a pointer never reads a parenthesized note containing "superseded" (edit kind 1's wording), so a Standing line resting on a bundle is checked by hand against what the bundle's later entries and other files reversed, as for any entry superseded in part. Every decisions file opens with one except the reference lists (below), a new file's cheapest moment to write it being its first entry; the files that predated the rule (C1's block landed 2026-10-19) were given theirs under #762, finished 2026-10-22, and the exempt list that held them while it ran is retired. `test_every_decisions_file_opens_with_standing_or_is_a_reference_list` fails on a file with no block that is not a reference list (the heading on a line of its own; the words quoted in prose do not count). The block stands ahead of every `## Log` heading; a `## Log` before it is a finding, and so is a heading that misspells `## Standing` (a trailing space, a different level), which would otherwise switch the pointer checks off. A decision of another rule's that this log is the record of stays a bullet naming its rule (decisions/C13's line for rules/C2, decisions/P9's for rules/S5); one recorded under another key is an `Elsewhere:` line pointing at that entry, not a bullet. A file of Declined, Excluded, Open, removal or mapping lines is a reference list instead, indexed under the Reference lists heading at the end of the index: it carries no block. A listed file carrying a block or taking a log's shape (a dated entry, a bullet opening with a tag and a date as `- (A) 2026-08-17 — …`, or a dated title) is a finding, as is a listed key outside the five fixed in `tests/v2/design_docs.py`'s `REFERENCE_LISTS` (2026-10-21, #762; fixed 2026-10-22). Whether a file in none of those shapes is a list or a log is a judgment the check cannot make, made in review. 3-0-reevaluations is not one: other files amend its lines without its saying so (decisions/cjk-comma-demotion scopes its FAMILY_COMMA line), the shape a Standing block's `Not standing:` and `Elsewhere:` lines are for. Five files split out on 2026-10-19 (C13, C14, P8, P9, T5) showed the rule was needed: none had a block, though C1's had landed the same day. **Before writing a line, look for what reversed it elsewhere.** A decision can be overturned by an entry in another file, which the log it stands in never mentions; #776 found two lines wrong that way, one while writing and one in review (P9's #578 note, reversed by decisions/S8's #602; P8's subset, widened by decisions/P3's #397), and #762's first batch (2026-10-21) found five entries whose heads did not say what had replaced part of them (decisions/S6's accepted `abdul Smith Jr Ma`, reversed by decisions/S7's #544; decisions/S4's `Doe, Jane nee Smith MA`, by decisions/M2's #601; decisions/H3's `Dr Smith V`, by decisions/S10's #718, caught only in that batch's PR review; decisions/P5's view comparison and decisions/S10's one-case `v`, by their own files' #614 and #721). So grep the whole folder for each issue and example a line cites, parse each example on the tree, and where a later decision reversed part of an entry mark that entry's head under 1, so the log says what Standing says. - **`Retired:`**, for a rule taken out of rules/: a line of its own in the rule's decisions file, opening `Retired:` and naming the date, the issue and what replaced the rule (`Retired: 2026-11-02 #800 — folded into C1.`), in the Standing block where the file has one, beside the dated entry that decided it. The ID is then never reused: `tests/v2/test_design_docs.py::test_no_rule_reuses_a_retired_id` refuses a rule file carrying it, a `Retired:` line in a file keyed by no rule ID, and a line that misspells `Retired:` (bold, a list marker or number, lowercase), which would switch the refusal off for its ID. No rule had been retired when the check landed (2026-10-20), so it guarded nothing on the tree until the first one is; its synthetic case is what shows it bites. - **Harvested entries** (content landed from a session's report) keep their provenance: a "measurements are that session's, spot-checked at landing" framing line, the contributor's measured-vs-remembered markings where they matter, and — the hard-won one — a flagged uncertainty is resolved by reading the source artifact, never by inference from the neighboring argument. - **`Declined:`** — proposals rejected WITH the evidence that killed them. Resolved-as-no is a decision; without a home for it, the next person re-derives the rejected proposal and its measurement. diff --git a/docs/design/decisions/parse-cost.md b/docs/design/decisions/parse-cost.md index 0d072437..00dce61a 100644 --- a/docs/design/decisions/parse-cost.md +++ b/docs/design/decisions/parse-cost.md @@ -33,7 +33,7 @@ Open: none in this file. - 2026-09-29 #561 -- SUPERSEDED IN PART: the 2026-09-26 bullet above, whose four conditions ("decorated itself", "keeps the generated `__init__`", no `__post_init__`, no `init=False` field) were not sufficient. `replace` builds through `obj.__class__(...)`, and a field copy skips every hook that call runs; four shapes met all four conditions and still parted: a validating `__new__`, a validating `__setattr__` on a non-frozen dataclass and a validating metaclass `__call__`, each giving `ValueError` from `replace` where an unguarded field copy built `value=-1`, and an `InitVar` with no default, which `replace` demands (a `ValueError` through 3.12, a `TypeError` from 3.13) and a field copy never sees. #561's first tightening (80554266) found only the first three, and by swapping the class's OWN `__dataclass_params__` for an inherited lookup it also let through an `__init__` borrowed from another dataclass under that dataclass's name, which the 2026-09-26 guard had refused; review caught both. So the guard is now a list of conditions a class must MEET (`_state._copy_refusals`, one reason string per unmet condition): decorated as a dataclass itself, frozen, an `__init__` generated for it (compiled from `` and named `.__init__`) that takes exactly its `init` fields, the default metaclass, no `__new__` above `object`, no `__post_init__`, no `init=False` field. It is a TRIPWIRE for a realistic edit to the three pipeline classes and says so, not a proof against any class: a forged `__qualname__` on a borrowed `__init__` in a class that is also decorated itself, or a `__class__` property, still passes, and `copy_with` copies nothing but `WorkToken`, `PendingAmbiguity` and `ParseState`, which meet every condition on 3.11 to 3.15. `_UNGUARDED_EFFECT` records for each row what `replace` builds, what an unguarded copy builds, and which conditions refuse it; `test_every_guard_condition_alone_refuses_a_recorded_shape` asserts that each condition is the only refusal of some row, so dropping one fails a test rather than contradicting this paragraph (checked by removing the `__new__` condition in a scratch copy: two failures). Call counts unchanged, the guard running at import only. - 2026-10-10 #624 (its rows superseded by 2026-10-10 #625) -- every row moves DOWN by 4: `parse` 308 → 304 on 3.11 and 289 → 285 on 3.12-3.15, `facade` 345 → 341 and 326 → 322, each measured on its own interpreter with `tools/perf/call_count.py` in isolated copies of 8e6e524a (#631's merge) and of the change, the import asserted (8e6e524a read exactly the recorded rows). `--modules` on 3.11: `_pieces.py` 51 → 49 and `_group.py` 25 → 23 -- group's leading-position scan, a generator asking `is_leading_title` per piece, gave way to `leading_titles`, which it already called, and `chain_lead`, which asks `is_prefix_piece` only of the titles. - 2026-10-10 #625 -- every row moves UP by 3: `parse` 304 → 307 on 3.11 and 285 → 288 on 3.12-3.15, `facade` 341 → 344 and 322 → 325, each measured on its own interpreter with `tools/perf/call_count.py` in isolated copies of 505d6dec (#632's merge) and of the change, the import asserted (505d6dec read exactly the recorded rows). `--modules` on 3.11: `_pieces.py` 49 → 52, `_group.py` 23 → 23. The reference name holds a particle run (`de la`), so it pays the merge's call and one more `merge_pieces` and `joined_tags`, the chain then merging `de la` with `Vega` where it had merged the three words at once; the first cut asked the title run and the lead in the merge and again in the read, +11, and group now asks the title run once and hands both on. Every name of three or more words read at the trailing site pays the call, +1 (`John Q. Smith` 202 → 203 on 3.11, warm); a name with a particle run pays the 3 (`Juan de la Vega` 230 → 233). -- 2026-10-04 #601 -- THE LINK PIN IS RE-POINTED FROM THE MAIDEN CLAUSE TO THE NAME, 2,301 → 2,678 on py3.11. #601 retired the clause's link exception, so the clause shape the pin had measured reached `_between_name_words` 0 times; the same 64-link run written in the name (`Jane Puig i … Soler`) reaches it once per link, and a per-link regression moves it by 64 against a 2% band of 53.6. Recorded 2026-10-22 for #762 from commit 1037c87c and `tests/v2/test_benchmark.py`'s comment beside the pin, the move having landed with no entry here. +- 2026-10-04 #601 (its pin superseded by 2026-10-06 #617) -- THE LINK PIN IS RE-POINTED FROM THE MAIDEN CLAUSE TO THE NAME, 2,301 → 2,678 on py3.11. #601 retired the clause's link exception, so the clause shape the pin had measured reached `_between_name_words` 0 times; the same 64-link run written in the name (`Jane Puig i … Soler`) reaches it once per link, and a per-link regression moves it by 64 against a 2% band of 53.6. Recorded 2026-10-22 for #762 from commit 1037c87c and `tests/v2/test_benchmark.py`'s comment beside the pin, the move having landed with no entry here. - 2026-10-06 #617 (its rows and link pin superseded by 2026-10-07 #619) -- every row moves DOWN by 10: `parse` 374 → 364 on 3.11 and 353 → 343 on 3.12-3.15, `facade` 411 → 401 and 390 → 380, each measured on its own interpreter with `PYTHONPATH= pythonX.Y tools/perf/call_count.py` against 83f4e914 (every row had sat inside its band before, the 3.12-3.15 ones at +5 over a 348 baseline). `--modules` puts all ten in group and the piece layer (`_group.py` + `_pieces.py` 142 → 132, every other module unchanged): P3's connective joins moved out of `_group_segment` into `_pieces.join_connectives`, which asks `is_conj_piece`, `is_title_piece` and `is_prefix_piece` directly where the loops had asked through one-line closures over them, two frames per test becoming one. The link pin moves with it, 2,678 → 2,479 for the 64-link name on 3.11 (`_LINK_BASELINE`), and there for two reasons, counted per function in #617's review: the closures (`conj` 133 frames, `title` and `prefix` one each) and the single-letter test, which had joined each connective's text through a generator (65 frames, one per link, more than the pin's whole band), less the one `join_connectives` frame. A deliberate move, not drift: the change was a refactor (decisions/P3, 2026-10-06). The reference rows hold no comma; a comma part holding a connective now pays the shared loop (`John Smith, Mr. and Mrs.` 262 → 269 on 3.11, warm), and one holding none skips it and costs what it did (`Smith, John` 161), the review having found the first cut charging every family-comma parse two frames per word of the part. THEN 39 MORE on every row, in the branch's /simplify pass: group ran the rootname count (about five frames a piece, its only reader the carve-out) and the shared loop over every segment of three or more pieces, connective or not, and over the corpora and the case table under three orders 0 of the 3,153 group calls with no connective left free changed a piece. Group now skips both where its frozen-set walk, which already reads every piece's `conjunction` tag, finds no connective free, as `_comma._reading_pieces` does: `parse` 364 → 325 on 3.11 and 343 → 304 on 3.12-3.15, `facade` 401 → 362 and 380 → 341, `_group.py` + `_pieces.py` 132 → 93, measured on each interpreter as above; the link pin does not move, every link in it joining. Per name on 3.11, warm: the Vega name 436 → 386, `Smith, John Quincy Adams Bob` 326 → 297, `Smith, John` and `John Smith, Mr. and Mrs.` unchanged at 161 and 269. No output moved over the #617 grids (decisions/P3). - 2026-10-07 #619 (its rows superseded by 2026-10-07 #614; #619 added 2026-10-22, the head having named no issue of its own: commit 343e268c landed through #619's merge, adf6da88) -- every row moves DOWN by 8 more: `_group_segment`'s last three one-line predicate closures, `prefix` over `is_prefix_piece`, `suffix` over `is_suffix_piece` and `marker` over `_is_maiden_marker_piece`, are inlined as direct calls, finishing what #617 did for `title`, `conj` and `merge`. Each closure was a frame of its own ahead of the predicate's, and the chain's two inner scans ask `prefix` once per piece they pass, so the cost grows with particle pieces (frame-budget question 1 in AGENTS.md). Measured with `PYTHONSAFEPATH=1 PYTHONPATH= pythonX.Y tools/perf/call_count.py`, each row on its own interpreter, parse/facade against 0c54662b: 3.11 325/362 → 317/354, 3.12, 3.13, 3.14 and 3.15 304/341 → 296/333. `--modules` on 3.11 puts all eight in `_group.py` (34 → 26), every other module unchanged. The whole of the drop is the closures, counted per function on 0c54662b (a profile hook counting `call` events whose code is named `prefix`, `suffix` or `marker` in `_group.py`): the reference name asked `prefix` 7 times and `suffix` once, `Juan` + `de la Vega ` × 8 + `Smith` asked 40 and 9 and reads 937 → 888 on 3.11 (919 → 870 on 3.12-3.15), and the 64-link name asked `prefix` once, so `_LINK_BASELINE` moves 2,479 → 2,478 on 3.11 and gains rows for 3.12-3.15 at 2,459 (2,460 on 0c54662b), each measured on its own interpreter and the first the pin has had beyond 3.11; `marker` is reached only by P5's bound-join decline and the reference name never asks it. BEHAVIOR IDENTICAL: the full suite passes, and the differential gate's report at all five baselines matches 0c54662b's line for line apart from the path header. - 2026-10-08 #620 (its rows superseded by 2026-10-10 #624) -- no row moves: `parse` 308 on 3.11 and 289 on 3.12-3.15, `facade` 345 and 326, measured as the #614 bullet below on 4278693d (#621's merge) and on the change alike. The 3.12-3.15 pin read 290/327 until this date: #614's /simplify took one frame off those interpreters (289/326 on 4278693d, measured 2026-10-08) and the pin stayed, inside its band, so the #614 bullet's 290 is the pin's number, not that tree's. `--modules` on 3.11 unchanged. The merged copy and its per-word exclusion test are gone, and `_chain_units` asks `chain_run_end` per particle unit and `_weighed` per word behind one; the chain itself calls `chain_run_end` with the particle test inline where it had called `is_prefix_piece` per piece, which pays for the rest. decisions/S2, this date.