diff --git a/docs/design/decisions/3-0-reevaluations.md b/docs/design/decisions/3-0-reevaluations.md index 3f422424..15488ce5 100644 --- a/docs/design/decisions/3-0-reevaluations.md +++ b/docs/design/decisions/3-0-reevaluations.md @@ -2,6 +2,46 @@ Promoted 2026-08-15 from session memory (Derek's 2026-07-30 ask; promotion approved 2026-08-15). Discipline when appending: mark each entry (A) "would decide differently without the shim" — real 3.0 work — or (B) "cited 1.4 parity but stands on its own", recorded so nobody re-litigates it. Append here whenever a design choice cites 1.4 parity or the shim as a load-bearing reason. Standing discipline for the removals themselves: every removal warns in a RELEASED version first — the rule established by the 1.3.0 eq/hash work (#223/#224), the reason the v1.4 milestone existed, and the reason FACADE-CONTRACT's "warning-free on 1.4" anchor works at all. 3.0's shim removals follow the same bridge. The CONVERSE rule, declined-with-reasoning twice in the migration design (2026-07-11): a bridge warning must WAIT when the replacement does not yet exist — 1.4 warned on neither shared-CONSTANTS mutation nor the subclass hooks, because users had no actionable response until 2.0, and #262's contract docs shipped in the same release (warn-while-documenting is a mixed message). +## Standing + +- The facade speaks v1's fields, `first` and `last`, over a core that speaks `given` and `family`, CJK names included; 3.0 can drop the aliasing — 2026-08-15. +- Render's default format is v1's, so an interpunct transcription renders space-joined (`str(HumanName("威廉·莎士比亚"))` is `'威廉 莎士比亚'`) and a caller reinstates the dot with a format of their own; 3.0's render defaults are a clean slate — 2026-08-15. +- A `.pyi` stub typing `Policy.__init__` wide and its attributes narrow is a 3.0 candidate, deferred from #334 for want of a signature list with its own sync test — 2026-08-15. +- Whether the vocabulary DATA modules keep `nameparser.config` as their home in 3.0 is open: the removal schedule names the whole package but enumerates only the five shim exports, and `Lexicon`'s field docs cross-reference `nameparser.config.particles` and its siblings (the maintainer note in `nameparser/config/__init__.py`) — 2026-08-15. +- `force_mixed_case_capitalization` is a second route into R5's override, reaching it only because the facade resolves the attribute when no per-call `force` is given (`HumanName('Shirley Maclaine')` with it on capitalizes to `'Shirley MacLaine'`); it is a removal candidate, nothing decided, and the bridge discipline in this file's preamble applies, a removal warning in a released version first — 2026-08-29. +- Pickle-guard layout breaks landing in a minor release are shim-era friction; the guarded raise stands on its own — 2026-08-15. +- Reading positionally where no script licenses family-first is the safe direction on its own, family-first being the marked case that needs affirmative evidence — 2026-08-15. +- The deprecation bridge's mechanics are the template for every remaining 2.x→3.0 shim: PEP 562 module `__getattr__` plus `__all__`, a warning per read location rather than per process, and the `TYPE_CHECKING` split — 2026-08-15. +- A scoped config override through contextvars (`with nameparser.config.use(c):`) stays parked for 3.0, the shared `CONSTANTS` surviving 2.x as the one config channel that reaches parses inside code a caller does not own — 2026-07-06. +- The 1.3.0-era legacy-pickle skip of `suffixes_prefixes_titles` rides in the shim's `__setstate__` and dies with it — 2026-08-16. +- An explicit family comma beats script for Latin text, a doctrine inherited from v1 that stands on its own; for CJK script it is a description, not a promise (Elsewhere, below) — 2026-08-15. +- Particle vocabulary is language judgement wearing a position heuristic, so a caller who knows a tradition should be able to opt into a tuned set, which needs packs to carry vocabulary overrides and not only additions; 3.0 work — 2026-08-17. +- Which single-letter connectives read as initials differs by language, a per-word policy that `conjunctions_ambiguous` carries without packs (`e` and `i`); a pack can add a member, so an `nl` pack could mark `y`, and cannot remove one (the next line) — 2026-08-18; 2026-09-20 #397. +- A locale pack's vocabulary is unioned onto the base and can remove nothing, so a caller removing a base entry, such as `e` from `conjunctions_ambiguous` for Portuguese, does it by hand; accepted for 2.x and part of 3.0's pack design — 2026-09-13, SHOULD A LOCALE PACK. +- The snapshot and `compare.py` read every corpus name under the default `Lexicon`, no corpus entry carrying a locale, so neither can show a pack's effect; the fix, a locale per corpus entry, is parked with the question above — 2026-09-13, THE DIFFERENTIAL HARNESS. +- The shim raises on a `capitalization_exceptions` value that does not spell its key's letters and digits, as the core `Lexicon` does; only where the raise lands for a v1 caller, at the first parse, is shim-era — 2026-09-23 #459. +- The shim's filter of v1 `Constants` entries dies with the shim, the core having no v1 reading to reproduce: it drops with one warning an entry v1 could never match, 1.4.0's two shipped `TITLES` typos silently, keeps a capitalized or edge-period key that v1 never matched and that matches now, an accepted widening, and drops a lone CJK full stop that 1.4.0 matched, its one departure from 1.4.0 (decisions/R4) — 2026-10-02 #541. + +Not standing: the harness bullet's "at all", overtaken for the agreement records, which score the CLDR renderings under their language's pack since #673 (issue #672) and the native-script names since #693, both recorded in tools/differential/README.md and in no decisions entry — 2026-09-13, THE DIFFERENTIAL HARNESS; the parking lot's requests-Session-style `Parser(constants)` API, overtaken by 2.0's `Parser(lexicon=..., policy=...)` — 2026-07-06. + +Elsewhere: the explicit-comma line narrowed to a description for CJK script — decisions/cjk-comma-demotion's 2026-09-01 #469, SCOPES the FAMILY_COMMA bullet entry; the 1.4 gate the harness-baseline bullet expected 3.0 to dissolve, already retired, `compare.py --baseline 1.4.0` answering the question when asked — decisions/differential-ledger's 2026-10-13 #664 entries; the vocabulary model packs are to narrow, a strict pack dropping other languages' readings — decisions/vocabulary-model's 2026-10-15/16 entry; what the shim's filter keeps and drops — decisions/R4's 2026-10-02 #541 and 2026-10-02 #582 entries. + +Open: #753 (phase 3, language packs, where the three pack bullets land). + +## Log + +- 2026-08-15 (its harness-baseline bullet superseded by decisions/differential-ledger's 2026-10-13 #664 retirement of the classified gate; its FAMILY_COMMA bullet superseded in part, narrowed for CJK script by decisions/cjk-comma-demotion's 2026-09-01 #469 entry) — THE LIST PROMOTED FROM SESSION MEMORY: the undated (A) and (B) bullets below, but for the three the lines after this one date (the parking lot, the legacy-pickle skip and `force_mixed_case_capitalization`). +- 2026-07-06 — THE CONFIG-MODEL PARKING LOT: the (A) bullet opening "Parking lot from the 2026-07-06 config-model discussion" below. +- 2026-08-16 — THE LEGACY-PICKLE SKIP, promoted from session memory by that day's sweep: the (B) bullet opening "The 1.3.0-era legacy-pickle property-key skip" below. +- 2026-08-17 — PARTICLE VOCABULARY AS LOCALE PACKS: the (A) bullet so headed below. +- 2026-08-18 (its open #383 and its pack-only requirement superseded in part by decisions/P8's 2026-09-13 #383 + #479 entry, whose `conjunctions_ambiguous` is the per-word policy without packs; its six single-letter conjunctions seven since decisions/P3's 2026-09-20 #397 entry) — LOCALE CONTROL OF P3's SINGLE-LETTER CARVE-OUT: the (A) bullet so headed below. +- 2026-08-29 — `force_mixed_case_capitalization` AS A SECOND ROUTE INTO R5's OVERRIDE: the (A) bullet opening with that attribute below. +- 2026-09-13 — SHOULD A LOCALE PACK BE ABLE TO REMOVE BASE VOCABULARY: the (A) bullet so headed below. +- 2026-09-13 — THE DIFFERENTIAL HARNESS CANNOT SEE A LOCALE PACK'S EFFECT: the (A) bullet so headed below. +- 2026-09-20 #397 — A SECOND INSTANCE OF THE SAME PER-LOCALE QUESTION: the (A) bullet so headed below. +- 2026-09-23 #459 — THE SHIM RAISES on a v1 config it once accepted: the (B) bullet so headed below. +- 2026-10-02 #541 — THE SHIM DROPS v1 entries v1 could never match: the (B) bullet opening THE SHIM DROPS below. + - (A) v1 field vocabulary at the facade boundary: CJK semantics squeeze into first/last through HumanName while the core speaks given/family. 3.0 could drop the aliasing entirely. - (A) Render's v1-inherited string_format defaults, including space-joined output for interpunct transcriptions (#298: str(HumanName("威廉·莎士比亚")) == "威廉 莎士比亚"; users reinstate the dot via custom format). 3.0 render defaults are a clean slate. - (A) The differential harness baseline is 1.4-on-PyPI: post-shim, the migration promise it verifies dissolves; the successor baseline is presumably last-2.x. Machinery survives; corpus contracts change. @@ -16,10 +56,10 @@ Promoted 2026-08-15 from session memory (Derek's 2026-07-30 ask; promotion appro - (A) Parking lot from the 2026-07-06 config-model discussion, 3.0-shaped by design: scoped config override via contextvars (`with nameparser.config.use(c):`, the decimal.localcontext pattern — fixes test pollution and config wars while keeping the shared default), and a requests-Session-style Parser(constants) API (rejected then as a full rewrite; the shared CONSTANTS survives 2.x because it is the only config channel that reaches parses inside code users don't own — #262's contract framing). - (B) The 1.3.0-era legacy-pickle property-key skip (suffixes_prefixes_titles in old blobs) rides in the shim's __setstate__ and dies with it. - (B) The FAMILY_COMMA doctrine (rule W3): inherited from v1's lastname-comma but correct on its own terms — an explicit comma is stronger evidence than script. -- (A) 2026-08-17 — PARTICLE VOCABULARY AS LOCALE PACKS. Raised while reversing the order-independence decision above: every argument in the leading-particle thread is a language judgement wearing a position heuristic. `de` and `do` lead surnames in Vietnamese, `von` is German, `dos` Portuguese, `das` both Portuguese and a borne Bengali surname — and the per-word comments in nameparser/config/particles.py (reading notes in nameparser/vocab/und.toml from 2026-10-18, and since #749 in the file of each language whose names the particle occurs in) cite exactly these traditions as their justification. Locale packs already exist (ru, tr_az, zh, ja) and already carry vocabulary and rotations, so particles are the obvious next tenant: a caller who knows the tradition could opt into a tuned set instead of the parser splitting the difference globally. Not 2.x work — it needs the packs to carry vocabulary OVERRIDES rather than additions, and it would give C-i's positional qualifier a per-locale answer rather than one global one. +- (A) 2026-08-17 — PARTICLE VOCABULARY AS LOCALE PACKS. Raised while reversing decisions/P1's order-independence decision (its 2026-08-16 order-precedence keystone entry, reversed by its 2026-08-17 entry; this pointer said "above" until 2026-10-22, from the single decisions.md this file was split out of): every argument in the leading-particle thread is a language judgement wearing a position heuristic. `de` and `do` lead surnames in Vietnamese, `von` is German, `dos` Portuguese, `das` both Portuguese and a borne Bengali surname — and the per-word comments in nameparser/config/particles.py (reading notes in nameparser/vocab/und.toml from 2026-10-18, and since #749 in the file of each language whose names the particle occurs in) cite exactly these traditions as their justification. Locale packs already exist (ru, tr_az, zh, ja) and already carry vocabulary and rotations, so particles are the obvious next tenant: a caller who knows the tradition could opt into a tuned set instead of the parser splitting the difference globally. Not 2.x work — it needs the packs to carry vocabulary OVERRIDES rather than additions, and it would give C-i's positional qualifier a per-locale answer rather than one global one. - (A) 2026-08-18 — LOCALE CONTROL OF P3's SINGLE-LETTER CARVE-OUT, and the sharper requirement it puts on the entry above: a pack would have to carry per-word POLICY, not just membership. Which single letters a tradition wants joined differs by language — Spanish writes "y" and would want "e" read as an initial, Portuguese wants the reverse, Dutch would want every single letter read as an initial. One global rule serves all three today: a single ALPHABETIC character in a name of fewer than four pieces stays a name word, and a bare Latin capital never joins (#383). It is v1's Google Code issue 11 fix ("john e smith") generalized to a vocabulary v1 did not have — the set now holds six single-letter conjunctions (y, e, и, і, й, و), and the carve-out reaches all of them because it counts characters, while the capital veto reaches only the Latin ones because it tests a Latin shape. That asymmetry is the visible seam and it is why #383 is open. A pack that could only ADD or REPLACE vocabulary could not express any of it; the join threshold and the initial veto would have to be per-word, per-pack values. Measured while documenting P3: "&" is single-character but not a letter, so it joins at any length ("Juan & Garcia" is one part while "Juan y Garcia" is three). - (A) 2026-09-13 — SHOULD A LOCALE PACK BE ABLE TO REMOVE BASE VOCABULARY? `Locale.lexicon` is UNIONED onto the base and never removes — the field's own comment says so — which makes `conjunctions_ambiguous` (decisions/P8, 2026-09-13) expressible by a pack in ONE DIRECTION ONLY: an `nl` pack can add "y", and a `pt` pack cannot remove "e", so a Portuguese caller has to write `Parser(lexicon=Lexicon.default().remove(conjunctions_ambiguous={"e"}))` by hand. Accepted for 2.x and parked here rather than filed, because 3.0 is when a wider set of locale packs gets designed and this is part of that work (Derek, 2026-09-13). It is the 2026-08-17 PARTICLE VOCABULARY AS LOCALE PACKS bullet's "overrides rather than additions" requirement meeting a second tenant, which is what makes it a shape question rather than one pack's inconvenience. - (A) 2026-09-13 — THE DIFFERENTIAL HARNESS CANNOT SEE A LOCALE PACK'S EFFECT AT ALL. `compare.py` builds one `Parser` per name ORDER and nothing else, and a `Case.locale` row projects into the corpora as a bare name string — no corpus row carries a locale key, verified over every `tools/differential/corpus*.jsonl` file on 2026-09-13 — so every corpus name is parsed under the DEFAULT `Lexicon` on both sides. A pack's effect on a name can therefore never diff, at any baseline. That is fine while packs only ADD vocabulary the default already carries most of, and stops being fine the moment the bullet above is answered yes. The shape of the fix is a `locale` carried per corpus ENTRY the way `shape` already carries an order, resolved at the baselines that ship the pack and skipped below them. Parked with the bullet above rather than filed, for the same reason: the two are one piece of work (Derek, 2026-09-13). -- (A) 2026-09-20 #397 — A SECOND INSTANCE OF THE SAME PER-LOCALE QUESTION, cross-referenced rather than parked again. Catalan/Polish `i` joins `conjunctions_ambiguous` beside `e`, so the marked subset now carries two letters and the 2026-08-18 bullet above — which counted six single-letter conjunctions and named the per-word-policy requirement — describes a set of seven, `i` included. Nothing in that bullet's argument moves: the subset IS the per-word policy the bullet asked for, made configurable without packs, and the pack-side half is still what the 2026-09-13 (A) bullet above asks, since a `ca` or `pl` caller still has to write `Lexicon.default().remove(conjunctions_ambiguous={"i"})` by hand. See `decisions/P3`'s 2026-09-20 bullet. +- (A) 2026-09-20 #397 — A SECOND INSTANCE OF THE SAME PER-LOCALE QUESTION, cross-referenced rather than parked again. Catalan/Polish `i` joins `conjunctions_ambiguous` beside `e`, so the marked subset now carries two letters and the 2026-08-18 bullet above — which counted six single-letter conjunctions and named the per-word-policy requirement — describes a set of seven, `i` included. Nothing in that bullet's argument moves: the subset IS the per-word policy the bullet asked for, made configurable without packs, and the pack-side half is still what the 2026-09-13 SHOULD A LOCALE PACK BE ABLE TO REMOVE BASE VOCABULARY? (A) bullet above asks (this pointer named only its date until 2026-10-22, which two bullets share), since a `ca` or `pl` caller still has to write `Lexicon.default().remove(conjunctions_ambiguous={"i"})` by hand. See `decisions/P3`'s 2026-09-20 bullet. - (B) 2026-09-23 #459 — THE SHIM RAISES on a v1 config it once accepted: a `capitalization_exceptions` value that does not spell its key's letters and digits raises `ValueError` at the first parse (decisions/R4, 2026-09-23). Cited against the shim's never-raise rule and taken anyway, so it is recorded here: the raise stands on its own — a case mask has no reading for a value that re-spells its word, and the core `Lexicon` raises the same way with no shim involved — and only where it LANDS for a v1 caller, at the first parse rather than at construction, is shim-era. 3.0 has nothing left to decide. - (B) 2026-10-02 #541 — THE SHIM DROPS, with a warning, every v1 `Constants` entry v1 could never match (decisions/R4, 2026-10-02). Cited from 1.4 parity, and it dies with the shim: the core `Lexicon` normalizes edge whitespace away and has no v1 reading to reproduce, so 3.0 has nothing left to decide. diff --git a/docs/design/decisions/README.md b/docs/design/decisions/README.md index cc9c9434..520e4a5e 100644 --- a/docs/design/decisions/README.md +++ b/docs/design/decisions/README.md @@ -11,7 +11,7 @@ Entry conventions: 2. **A factual error is corrected in place**, with a note saying what was wrong — a date, an attribution, a measurement or a pointer. decisions/C1's 2026-07-12/13 entry, whose parenthetical moved a misdated credit, is the shape. 3. **Compaction, on a stated trigger.** A superseded entry is cut to a tombstone: its date, issue, verdict line and `SUPERSEDED by`, plus whatever it recorded that the superseding entry did not replace — a gap it found standing before its change, or an option it declined that the reversal does not answer, stays beneath the tombstone (decisions/T4's #731); a superseded passage inside a live entry is cut to its decision and what replaced it. A measurement paragraph (what moves, cost, blast radius) is cut to one line naming its population, counts, baseline and PR once its change has shipped in a release. The full text stays in git and in the PR the entry cites. #728 is the worked compaction. A harvested entry (below) keeps its framing line through compaction, a tombstone included, and its measured-vs-remembered markings wherever its measurements survive. A trap #759's review found: a one-line replacement carries each count with the classification the old text gave that count and no other (decisions/S2's #614 line had given a whole grid the classification measured on one slice of it). 4. **A per-word fact that moves into the vocabulary's TOML is REPLACED by a query that finds it** (`uv run python tools/vocab/query.py --issue 661`, `--excluded generation`), not left beside a dated "moved" note, which would keep two copies to drift. Only the per-word sentences go; the decision, its criterion and its measurement stay as written. Sanctioned since 2026-10-18 (decisions/vocabulary-model). -- **`## Standing`**, at the head of a file: the one part edited freely. One line per decision in force, in the present tense, ending with the entry that decided it by its exact date and the first issue its head names (`— 2026-10-06 #613`), or by its date alone where the head names none; where several entries share that date and issue, the pointer goes on with the words its entry's head continues with (`— 2026-09-14 (Derek)`, `— 2026-10-18 #718 review`), the longest such continuation winning and a date inside those words being part of them; a pointer with no words names the entry whose head ends there, and one whose words continue no head there fails, since nothing tells a typo from prose, so prose after such a pointer goes after a `.` or `;` (the words end there, or where the next pointer's date begins); where two heads are the same words, the pointer goes on with the words its entry's text opens with (`— 2026-09-13 #383/#479, AMENDS`), a key rather than prose, and so it does where another head of that date and issue continues past those words (decisions/M3's `— 2026-08-26 #335, the MECHANISM`, beside its `#335 (M3, …)` head); and a pointer still naming more than one entry fails as ambiguous (a parenthesized supersession note in a head is not read), plus a `Not standing:` line for entries whose plan was carried out or whose schedule events overtook, its pointers read and failing as a Standing line's do except that they may name a superseded entry — a decision reversed by a later DECISION is marked under 1 instead and an `Open:` line for each open question. It indexes the log and is never a decision itself: a Standing line with no entry behind it, or only a superseded one, fails `tests/v2/test_design_docs.py::test_every_standing_line_points_at_a_live_entry`, and changing a decision is still a new entry, which updates Standing in the same diff. The entries follow under `## Log`. An entry is a log bullet opening with its date, bare or in bold (`- **2026-10-14 #679 — …**`); a file's title founds no entry, so a bundle recorded as undated bullets under a dated title gains, when its block is written, one dated line opening its Log and naming the bundle, its verdict the title's words (`- 2026-09-10 #322/#323 — WHICH CHARACTERS ARE A PERIOD ON CJK TEXT: the bullets below.`). It dates what the title already dates, so it adds no decision and edits no entry. Where a later decision reversed part of the bundle, its head says which, under edit kind 1. That note is for the reader: a pointer never reads a parenthesized note containing "superseded" (edit kind 1's wording), so a Standing line resting on a bundle is checked by hand against what the bundle's later entries and other files reversed, as for any entry superseded in part. Every decisions file created after C1's block landed (2026-10-19) opens with one, the cheap moment to write it being the file's first entry; the older files are listed in `tests/v2/decisions_without_standing.json`, decisions/C12 and decisions/vocabulary-model among them though they too were created that day, before the rule. `test_a_decisions_file_off_the_list_opens_with_standing` fails on a file off that list with no block (the heading on a line of its own; the words quoted in prose do not count), on a listed file that has gained one, on a listed key no file carries, and on a listed key the list did not hold when the rule landed (a fixed set in `tests/v2/design_docs.py`, its size asserted), so the list never holds a key it did not start with: a new file can join it neither alone nor in exchange for one that leaves. The block stands ahead of every `## Log` heading; a `## Log` before it is a finding, and so is a heading that misspells `## Standing` (a trailing space, a different level), which would otherwise switch the pointer checks off. A decision of another rule's that this log is the record of stays a bullet naming its rule (decisions/C13's line for rules/C2, decisions/P9's for rules/S5); one recorded under another key is an `Elsewhere:` line pointing at that entry, not a bullet. Whether the older files get one is #762. A file of Declined, Excluded, Open, removal or mapping lines is a reference list instead, indexed under the Reference lists heading at the end of the index: it carries no block. A listed file carrying a block, taking a log's shape (a dated entry, a bullet opening with a tag and a date as `- (A) 2026-08-17 — …`, or a dated title) or sitting on the exempt list is a finding, as is a listed key the exempt list did not hold when the rule landed (2026-10-21, #762). Whether a file in none of those shapes is a list or a log is a judgment the check cannot make, made in review. 3-0-reevaluations is not one: other files amend its lines without its saying so (decisions/cjk-comma-demotion scopes its FAMILY_COMMA line), the shape a Standing block's `Not standing:` and `Elsewhere:` lines are for. Five files split out on 2026-10-19 (C13, C14, P8, P9, T5) showed the rule was needed: none had a block, though C1's had landed the same day. **Before writing a line, look for what reversed it elsewhere.** A decision can be overturned by an entry in another file, which the log it stands in never mentions; #776 found two lines wrong that way, one while writing and one in review (P9's #578 note, reversed by decisions/S8's #602; P8's subset, widened by decisions/P3's #397), and #762's first batch (2026-10-21) found five entries whose heads did not say what had replaced part of them (decisions/S6's accepted `abdul Smith Jr Ma`, reversed by decisions/S7's #544; decisions/S4's `Doe, Jane nee Smith MA`, by decisions/M2's #601; decisions/H3's `Dr Smith V`, by decisions/S10's #718, caught only in that batch's PR review; decisions/P5's view comparison and decisions/S10's one-case `v`, by their own files' #614 and #721). So grep the whole folder for each issue and example a line cites, parse each example on the tree, and where a later decision reversed part of an entry mark that entry's head under 1, so the log says what Standing says. +- **`## Standing`**, at the head of a file: the one part edited freely. One line per decision in force, in the present tense, ending with the entry that decided it by its exact date and the first issue its head names (`— 2026-10-06 #613`), or by its date alone where the head names none; where several entries share that date and issue, the pointer goes on with the words its entry's head continues with (`— 2026-09-14 (Derek)`, `— 2026-10-18 #718 review`), the longest such continuation winning and a date inside those words being part of them; a pointer with no words names the entry whose head ends there, and one whose words continue no head there fails, since nothing tells a typo from prose, so prose after such a pointer goes after a `.` or `;` (the words end there, or where the next pointer's date begins); where two heads are the same words, the pointer goes on with the words its entry's text opens with (`— 2026-09-13 #383/#479, AMENDS`), a key rather than prose, and so it does where another head of that date and issue continues past those words (decisions/M3's `— 2026-08-26 #335, the MECHANISM`, beside its `#335 (M3, …)` head); and a pointer still naming more than one entry fails as ambiguous (a parenthesized supersession note in a head is not read), plus a `Not standing:` line for entries whose plan was carried out or whose schedule events overtook, its pointers read and failing as a Standing line's do except that they may name a superseded entry — a decision reversed by a later DECISION is marked under 1 instead and an `Open:` line for each open question. It indexes the log and is never a decision itself: a Standing line with no entry behind it, or only a superseded one, fails `tests/v2/test_design_docs.py::test_every_standing_line_points_at_a_live_entry`, and changing a decision is still a new entry, which updates Standing in the same diff. The entries follow under `## Log`. An entry is a log bullet opening with its date, bare or in bold (`- **2026-10-14 #679 — …**`); a file's title founds no entry, so a bundle recorded as undated bullets under a dated title gains, when its block is written, one dated line opening its Log and naming the bundle, its verdict the title's words (`- 2026-09-10 #322/#323 — WHICH CHARACTERS ARE A PERIOD ON CJK TEXT: the bullets below.`). It dates what the title already dates, so it adds no decision and edits no entry. A file with no dated title dates its bundle line by the date its preamble gives the bullets. A bullet the bundle line's date does not fit, because its own text dates its decision otherwise (`- (A) 2026-08-17 — …`, `Amended 2026-08-23`; a date it gives for something else, such as when a code change landed, is not one) or, where its text dates none, because `git log -S` over `docs/design/` (not over the file, which every bullet entered on 2026-10-17, when #713 split the one decisions.md into these files) shows it entering on a later day than the bundle was written, its earliest hit (`--reverse`) on a phrase short enough to have stood on one line before the 2026-08-16 unwrap, 5e2ae60b, a first hit there meaning the phrase crossed a wrapped line (decisions/3-0-reevaluations' legacy-pickle bullet, added the day after its list was promoted; a bullet written the day its bundle was, as decisions/comma-suffix-arc's Declined bullets were, keeps the bundle line's date even where that date is older), gains a dated line of its own beside the bundle line, dated by that decision date or by the author date of the commit that added it, naming the bullet by the words it opens with, since a pointer reads only a bullet that opens with its date (decisions/3-0-reevaluations and decisions/comma-suffix-arc, 2026-10-22). Where a later decision reversed part of the bundle, its head says which, under edit kind 1. That note is for the reader: a pointer never reads a parenthesized note containing "superseded" (edit kind 1's wording), so a Standing line resting on a bundle is checked by hand against what the bundle's later entries and other files reversed, as for any entry superseded in part. Every decisions file created after C1's block landed (2026-10-19) opens with one, the cheap moment to write it being the file's first entry; the older files are listed in `tests/v2/decisions_without_standing.json`, decisions/C12 and decisions/vocabulary-model among them though they too were created that day, before the rule. `test_a_decisions_file_off_the_list_opens_with_standing` fails on a file off that list with no block (the heading on a line of its own; the words quoted in prose do not count), on a listed file that has gained one, on a listed key no file carries, and on a listed key the list did not hold when the rule landed (a fixed set in `tests/v2/design_docs.py`, its size asserted), so the list never holds a key it did not start with: a new file can join it neither alone nor in exchange for one that leaves. The block stands ahead of every `## Log` heading; a `## Log` before it is a finding, and so is a heading that misspells `## Standing` (a trailing space, a different level), which would otherwise switch the pointer checks off. A decision of another rule's that this log is the record of stays a bullet naming its rule (decisions/C13's line for rules/C2, decisions/P9's for rules/S5); one recorded under another key is an `Elsewhere:` line pointing at that entry, not a bullet. Whether the older files get one is #762. A file of Declined, Excluded, Open, removal or mapping lines is a reference list instead, indexed under the Reference lists heading at the end of the index: it carries no block. A listed file carrying a block, taking a log's shape (a dated entry, a bullet opening with a tag and a date as `- (A) 2026-08-17 — …`, or a dated title) or sitting on the exempt list is a finding, as is a listed key the exempt list did not hold when the rule landed (2026-10-21, #762). Whether a file in none of those shapes is a list or a log is a judgment the check cannot make, made in review. 3-0-reevaluations is not one: other files amend its lines without its saying so (decisions/cjk-comma-demotion scopes its FAMILY_COMMA line), the shape a Standing block's `Not standing:` and `Elsewhere:` lines are for. Five files split out on 2026-10-19 (C13, C14, P8, P9, T5) showed the rule was needed: none had a block, though C1's had landed the same day. **Before writing a line, look for what reversed it elsewhere.** A decision can be overturned by an entry in another file, which the log it stands in never mentions; #776 found two lines wrong that way, one while writing and one in review (P9's #578 note, reversed by decisions/S8's #602; P8's subset, widened by decisions/P3's #397), and #762's first batch (2026-10-21) found five entries whose heads did not say what had replaced part of them (decisions/S6's accepted `abdul Smith Jr Ma`, reversed by decisions/S7's #544; decisions/S4's `Doe, Jane nee Smith MA`, by decisions/M2's #601; decisions/H3's `Dr Smith V`, by decisions/S10's #718, caught only in that batch's PR review; decisions/P5's view comparison and decisions/S10's one-case `v`, by their own files' #614 and #721). So grep the whole folder for each issue and example a line cites, parse each example on the tree, and where a later decision reversed part of an entry mark that entry's head under 1, so the log says what Standing says. - **`Retired:`**, for a rule taken out of rules/: a line of its own in the rule's decisions file, opening `Retired:` and naming the date, the issue and what replaced the rule (`Retired: 2026-11-02 #800 — folded into C1.`), in the Standing block where the file has one, beside the dated entry that decided it. The ID is then never reused: `tests/v2/test_design_docs.py::test_no_rule_reuses_a_retired_id` refuses a rule file carrying it, a `Retired:` line in a file keyed by no rule ID, and a line that misspells `Retired:` (bold, a list marker or number, lowercase), which would switch the refusal off for its ID. No rule had been retired when the check landed (2026-10-20), so it guarded nothing on the tree until the first one is; its synthetic case is what shows it bites. - **Harvested entries** (content landed from a session's report) keep their provenance: a "measurements are that session's, spot-checked at landing" framing line, the contributor's measured-vs-remembered markings where they matter, and — the hard-won one — a flagged uncertainty is resolved by reading the source artifact, never by inference from the neighboring argument. - **`Declined:`** — proposals rejected WITH the evidence that killed them. Resolved-as-no is a decision; without a home for it, the next person re-derives the rejected proposal and its measurement. diff --git a/docs/design/decisions/comma-suffix-arc.md b/docs/design/decisions/comma-suffix-arc.md index 8f7532eb..3d8d8683 100644 --- a/docs/design/decisions/comma-suffix-arc.md +++ b/docs/design/decisions/comma-suffix-arc.md @@ -3,9 +3,28 @@ #291 was filed 2026-07-26 out of the 2.0 vocabulary cleanup, with its decline-by-measurement evidence in the issue body; "z domu" folded into it 2026-07-27 (see Excluded, MAIDEN_MARKERS). Intent for #291 and #296 is settled by the approved bundle spec; #296 (with #325) SHIPPED 2026-08-23 as the bundle's first three commits, with one deviation from the audit table recorded at decisions/C8 ('md' keeps dual membership — bare 'Md' is Muhammad); #291 was UNSHIPPED then, carried as a deviates: marker on C1 (#291 closed 2026-08-24; the marker came off with #564 in 2.4). The arc's bookkeeping: #291/#296 moved milestone v2.1 → v2.2 on 2026-08-01 (with #289/#293); the 2026-07-30 design doc was amended in place 2026-08-01 (A1–A6); its durable content is HERE rather than in the gitignored doc; #316 (trailing titles) was filed the same day, carrying the esq cleanup as a Related section. +## Standing + +- The multi-word suffix entries v1 shipped unmatchable were removed in 2.0 rather than split into single-word entries, which would cost `Smith, A.P.` its given initials and `John Leed` and `Mary Nicet` their family names, all three reading as before today — 2026-07-26 #291. +- A multi-word credential is reached from its component words by the run predicate, so the shipped vocabulary carries no phrase entries and a caller adds a missing component word to a `Lexicon`: `John Smith, MD PhD` → suffix 'MD PhD', `John Smith, PSM I` → suffix 'PSM I'. `SUFFIX_PHRASES` and segment-level matching are declined, and amendment A6's glued-peel question with them, there being no phrase unit for the peel to step over — 2026-08-23 #291. +- Multi-word vocabulary merges where the render tag is applied, not where the role is assigned, since the suffix view comma-joins suffix words not tagged `joined`: `John Smith, LEED AP` renders suffix 'LEED AP' — 2026-07-26 #291. +- An unlisted abbreviation ending a name gets no structural fallback, the corpora being blind to it, and reads as a name word (`John Smith Xyz.` → family 'Xyz.'); written as two or more dotted chunks it is the exception, S9's shape class (`John Smith X.Y.Z.` → suffix 'X.Y.Z.') — 2026-07-26 #291. + +Not standing: plan amendment 1's repair, carried out by #296's first commit — 2026-08-01. + +Elsewhere: the audit table's dispositions as shipped, `md` kept a dual and `ms` and `sa` kept out of the ambiguous set — decisions/C8's 2026-08-23 #296 and 2026-08-23 #428 Standing lines, the second also holding the repair's condition, no name word; the trailing slot read by vocabulary, a period-marked title word there a title (`John Smith Prof.` → title 'Prof.') — decisions/H5's 2026-09-08 #316 entry; the dotted shape class — decisions/S9's 2026-09-14 (Derek) entry headed #516's DOTTED HALF IS A SWITCH. + +Open: none in this file. + +## Log + +- 2026-07-26 #291/#296/#316 (its audit table's `ms` and `sa` joining the ambiguous set superseded by decisions/C8's 2026-08-23 (review of PR #428, four reviewers) entry; its trailing-abbreviation decline superseded for the dotted spelling by decisions/S9's 2026-09-14 (Derek) entry headed #516's DOTTED HALF IS A SWITCH) — THE COMMA-SUFFIX ARC, 2026-07-26 → 2026-08-01: the paragraph, the plan amendment and the Declined entries below. + The approved 19-word TITLES∩suffix audit table (2026-07-30/08-01, intent for the unshipped bundle): jr, junior, phd, md, do and se DROP from TITLES (se: Structural Engineer is the PE/SE post-nominal pair, no prenominal convention — v1 residue); dr drops from SUFFIX_WORDS and sra from the suffix sets (v1 residue); sr, lt, cpl, cpt, cpo, csm, sgm, ra and vc KEEP dual membership (position decides); ms and sa keep dual membership AND join the ambiguous set (the periods gate). Derek's framing thesis for the whole table: v1's sets encoded where the v1 PARSER needed words to be, not where words can occur. Amendment A6: once SUFFIX_PHRASES ships, the glued peel's token-level post-nominal test cannot step over a phrase segment ("김민준씨, LEED AP" will not peel) — #291 owns deciding that, with a case row either way. -- 2026-08-01 (plan amendment 1, Derek's call: "ship + repair the merge") — dropping dr/ms from the suffix vocabulary flips segment() to FAMILY_COMMA and degrades "John Smith, Dr." to family="John Smith" (mechanisms.md#VOCABULARY-FEEDS-STRUCTURE is this shape); the approved repair, ordered FIRST in the bundle so no commit introduces a regression a later one heals: assign segment 0 positionally when segment 1 is nothing but titles. +- 2026-08-01 (plan amendment 1, Derek's call: "ship + repair the merge") (its condition superseded by decisions/C8's 2026-08-23 #428 entry, which reads no name word where this says nothing but titles) — dropping dr/ms from the suffix vocabulary flips segment() to FAMILY_COMMA and degrades "John Smith, Dr." to family="John Smith" (mechanisms.md#VOCABULARY-FEEDS-STRUCTURE is this shape); the approved repair, ordered FIRST in the bundle so no commit introduces a regression a later one heals: assign segment 0 positionally when segment 1 is nothing but titles. + +- 2026-08-23 #291 — #291 CLOSED WORKING AS DESIGNED: the `Amended 2026-08-23` paragraph under the first Declined entry below. Declined: diff --git a/docs/design/decisions/deviates-registry.md b/docs/design/decisions/deviates-registry.md index f9a54849..2f5ddc01 100644 --- a/docs/design/decisions/deviates-registry.md +++ b/docs/design/decisions/deviates-registry.md @@ -1,4 +1,12 @@ # deviates-registry — packs stay pure data (option C) +## Standing + +- A locale pack declares its deviations from the defaults as a module-level `DEVIATES` callable, acceptance metadata kept off `Locale`, whose value equality and pickling a callable field would break; `test_registry_is_the_pack_contract` fails unless every registered pack ships `DEVIATES` and a rotator list, which is what lets mechanisms.md#LOCALE-PACKS-PURE-DATA promise packs no code paths of their own. A field on `Locale` is to be revisited only if a runtime consumer of the declaration, such as an `explain()`, appears, and none has — 2026-07-18. + +Open: none in this file. + +## Log + - 2026-07-18 (d4aaafa; the DEVIATES design note) — a `deviates` field ON Locale was rejected: callables break value equality and pickle-by-reference, and DEVIATES is acceptance metadata, not runtime behavior (revisit only if a runtime consumer like explain() appears). The REGISTRY became the contract instead: test_registry_is_the_pack_contract fails structurally unless every registered pack ships DEVIATES and its rotator list. This is why LOCALE-PACKS-PURE-DATA can promise "no code paths of their own" and mean it. diff --git a/docs/design/decisions/differential-ledger.md b/docs/design/decisions/differential-ledger.md index cf53f63d..179c813c 100644 --- a/docs/design/decisions/differential-ledger.md +++ b/docs/design/decisions/differential-ledger.md @@ -4,7 +4,20 @@ The differential ledgers were the classified gate 2.0 shipped behind: `tools/dif Archived 2026-10-19 under rule 3 of the decisions folder's entry conventions (#759): eleven arcs of entries, about 33,000 words, were cut to the tombstones below, every one superseded by the retirement. The full text is in git — `git show fe85bb3d:docs/design/decisions/differential-ledger.md` — and in the PRs each tombstone names. The retirement entry is the one decision in force and stands in full; the passages other entries cite stand as one line each, and three Declined entries about surviving code stand as written. -## differential-ledger, the arcs before retirement (2026-07-24 to 2026-09-05) +## Standing + +- `tools/differential/snapshot.jsonl` records every corpus name's reading, one line per (name, order): the seven role fields, the ambiguity kinds and `initials()`, and what `HumanName` reads differently, under `facade`; `tests/v2/test_snapshot.py` fails on drift and prints the moves grouped by changed field, and a change that moves readings re-records, so the PR diff of the snapshot is the list of names it moved. Nothing is classified, and a reviewer can approve a large diff without reading it, the grouping being the mitigation; the ledgers, their guards and the contract/radar tiers are gone, trimming them having been declined — 2026-10-13 #664, a committed snapshot. +- `compare.py` stays, unclassified: it runs a released wheel against the tree behind the version tell, the tree-side path check and a stripped `PYTHONPATH`, prints the snapshot's grouped report and exits 0. It answers how a release tagged before the snapshot read the corpus names, and the 1.4 promise is a question asked of it (`--baseline 1.4.0`), not a gate; for a later release `snapshot.py --since ` is the cheaper answer — 2026-10-13 #664, `compare.py` stays. +- The CLDR agreement ratchet stays beside the snapshot, answering agreement with a corpus's stated fields rather than stability (`tests/v2/test_cldr_agreement.py`) — 2026-10-13 #664, the CLDR agreement ratchet. +- `Policy`'s fields are annotated with what they store, not widened to every input the constructor accepts, since every reader would see the union; a `.pyi` stub typing `__init__` wide and the attributes narrow is a 3.0 candidate (decisions/3-0-reevaluations) — 2026-07-24 #332. +- The tree side of a comparison must import this checkout's source package, `is_relative_to( / "nameparser")` and not anywhere under the repository, whose `.venv/`, `build/` and `dist/` can hold a released wheel; `compare.py` asks `snapshot.check_source` for it — 2026-07-24 #332. +- `DEFAULT_BASELINE` rotates at every release as a step of the AGENTS.md release checklist, not as an issue — 2026-07-24 #332. + +Open: none in this file. + +## Log + +### differential-ledger, the arcs before retirement (2026-07-24 to 2026-09-05) - 2026-07-24 to 2026-08-07, the tooling arc (#332, #333, #334) — harvested 2026-08-15 from the release-arc session; measurements are that session's, spot-checked at landing. The two-corpus design (corpus_issues.jsonl beside the v1-test-bank corpus, since v1 test banks are blind to anything 2.0 added), ledger fields named by `Role`, and the sentinel-set over-match check (mechanisms.md#SENTINEL-SET-OVER-MATCH-CHECK). SUPERSEDED by 2026-10-13 #664 — except three of its Declined entries, below, which are about code and process that survive the ledgers. - 2026-08, the dormancy arc (#328–#376) — measurements are that session's, the machinery verified present at landing. `[[never]]` exclusions sorted ahead of the rules, so an exclusion only ever removes a name; a rule that claims nothing is dormant and is refused. SUPERSEDED by 2026-10-13 #664 — @@ -29,7 +42,7 @@ Declined (the tooling arc's, harvested as above, kept because they are not about - `_check_tree` as is_relative_to(REPO_ROOT) (2026-08-05 #332) — accepts .venv/, build/ and dist/ copies inside the repo; the invariant is "is the source package", so the predicate is is_relative_to(REPO_ROOT / "nameparser"). - An issue for rotating DEFAULT_BASELINE (2026-08-07) — it recurs every release; it lives in the AGENTS.md release checklist beside the VERSION bump instead (#333 must land first). -## differential-ledger, retired for a snapshot (2026-10-13, #664) +### differential-ledger, retired for a snapshot (2026-10-13, #664) The ledgers' last arc. They were built to show that 2.0 changed nothing against 1.4 except fixes the release log lists, and they did; as a standing check every later behavior change had to be classified once per ledger, and the guards policing the ledgers outgrew the ledgers. diff --git a/docs/design/decisions/parse-cost.md b/docs/design/decisions/parse-cost.md index e11d648b..331dcac3 100644 --- a/docs/design/decisions/parse-cost.md +++ b/docs/design/decisions/parse-cost.md @@ -1,8 +1,25 @@ # parse-cost — what a parse is allowed to cost -Every number below is a py3.11 measurement of 2026-08-31, recomputable with `uv run python tools/perf/call_count.py` (add `--modules`, `--stages`, or `--against `). That harness is in the tree BECAUSE of this entry: its first draft took figures across several sessions with throwaway scripts on whichever interpreter was to hand, and published a module table that spliced one interpreter's before-column onto another's after-column. Four of its eight module numbers were wrong, its four stage numbers were 2x (2000-parse totals labelled per 1000, which its own arithmetic contradicted — four stages summing past the whole parse), and it blamed the wrong PR. Quote nothing here without the interpreter beside it. +The 2026-08-31 entry's numbers were measured that date, on py3.11 wherever no other interpreter stands beside them (this said "every number below is a py3.11 measurement of 2026-08-31" until 2026-10-22, when that entry and later ones had long quoted other interpreters and dates), recomputable with `uv run python tools/perf/call_count.py` (add `--modules`, `--stages`, or `--against `). That harness is in the tree BECAUSE of this entry: its first draft took figures across several sessions with throwaway scripts on whichever interpreter was to hand, and published a module table that spliced one interpreter's before-column onto another's after-column. Four of its eight module numbers were wrong, its four stage numbers were 2x (2000-parse totals labelled per 1000, which its own arithmetic contradicted — four stages summing past the whole parse), and it blamed the wrong PR. Quote nothing here without the interpreter beside it. -- 2026-08-31 #475 — the benchmark bounds FUNCTION CALLS per parse, not seconds, as a per-interpreter BAND of ±2%. A loose 5s wall-clock backstop stays over BOTH entry points for what frame counts cannot see: a compiled regex that starts backtracking emits no `call` event, and neither does C-level work. (A quadratic inside a comprehension or generator IS visible — `call` fires once per generator resume.) +## Standing + +- The benchmark bounds the function calls a parse makes, not its seconds, as a band of ±2% around a row per interpreter (`_CALL_BASELINE` and `_BAND` in `tests/v2/test_benchmark.py`), so a drop trips it as a rise does, and an interpreter with no row fails with its own number; a 5s wall-clock backstop over 1000 parses at both entry points stays for what frame counts cannot see, a backtracking regex and C-level work — 2026-08-31 #475. +- Raising or lowering a row is a decision, appended here with the interpreter and the harness invocation, every number coming from `tools/perf/call_count.py` and quoted with its interpreter — 2026-08-31 #475. +- The rows stand at `parse` 307 and `facade` 344 on 3.11, 288 and 325 on 3.12 to 3.15, as the last move left them, and `call_count.py` reads 307/344 on 3.11 (2026-10-22) — 2026-10-10 #625. +- The 64-link pin, `_LINK_BASELINE`, stands at 2,478 on 3.11 and 2,459 on 3.12 to 3.15 under the same band — 2026-10-07 #619. +- Collapsing `post_rules`' three role-index scans into one pass, 6 calls on 3.11, was tried and reverted on a timing harness that could not see it, and is left to reconsider against the band; the three scans stand — 2026-08-31 #475. +- The stages copy their state with `_state.copy_with`, a field copy, rather than `dataclasses.replace`, for `WorkToken`, `PendingAmbiguity` and `ParseState` only, and mypy sees it as `replace` imported under that name, keeping the dataclass plugin's checks at every call site — 2026-09-26 #546. +- The field copy is guarded by a list of conditions a class must meet (`_state._copy_refusals`), a tripwire for a realistic edit to those three classes and not a proof against any class; `_UNGUARDED_EFFECT` records what each refused shape would build, and each condition is the only refusal of some row — 2026-09-29 #561. +- Group asks its piece predicates by direct call, with no one-line closure between, and skips the rootname count and the shared connective loop where its tag walk finds no connective free to join — 2026-10-06 #617; 2026-10-07 #619. + +Not standing: #475's pointer to #424's peel walks as where to recover the cycle's growth, taken up by #614's single read of the trailing run — 2026-08-31 #475, 2026-10-07 #614. + +Open: none in this file. + +## Log + +- 2026-08-31 #475 (its rows superseded by 2026-09-26 #546) — the benchmark bounds FUNCTION CALLS per parse, not seconds, as a per-interpreter BAND of ±2%. A loose 5s wall-clock backstop stays over BOTH entry points for what frame counts cannot see: a compiled regex that starts backtracking emits no `call` event, and neither does C-level work. (A quadratic inside a comprehension or generator IS visible — `call` fires once per generator resume.) WHY THE CLOCK HAD TO GO. The 1.0s bound failed four times across #466 and #474, always on CI's 3.12 job, always between 1.01s and 1.08s, with master re-running green each time. Three local methods disagreed with CI and with each other: uninstrumented ~1% branch-over-parent (inside the parent's own spread), coverage-instrumented indistinguishable, and the whole benchmark file under `--cov` made the branch FASTER. The same harness measured `origin/master` at 94-96ms and again at 89-92ms twenty minutes later. A check that cannot separate a 1% change from a busy runner does not fail safe; it fails expensively. WHY A BAND AND NOT A CEILING, which is the correction the review forced. A ceiling with headroom is ANOTHER threshold that happens to break — the failure this replaced. The first draft set 470 against a 408 baseline, and measured, that catches nothing smaller than +61 calls on 3.11 and +84 on 3.12: it would have missed the very regression it was built for on four of five interpreters. The cycle's +67 arrived across a dozen PRs at roughly five calls each, and no ceiling loose enough to be safe can see five. A ±2% band trips at +5, verified by injection. A DROP trips it too, which is intended: an unexplained fall is as much a signal as a rise. PER INTERPRETER, because the count is not machine-independent — 410 on 3.11, 388 on 3.12, 406 on 3.13-3.15 for `parse`. PEP 709 inlined the comprehension frames 3.11 counts and 3.13 added others back. The first draft claimed "the same number on any machine" and took its baseline on 3.11 by luck: had it been taken on 3.12, the 3.11 job would have gone red with nothing in the file explaining why. A version with no row fails loudly with its own number rather than passing unguarded. @@ -12,12 +29,12 @@ Every number below is a py3.11 measurement of 2026-08-31, recomputable with `uv ONE REPAIR WAS TRIED AND REVERTED: collapsing `post_rules`' three separate role-index scans into one pass. It saves **6 calls per parse** on 3.11 (the first draft said one, which is not even reachable — `_idx` is a one-line list comprehension, so each removed call frees two frames). Reverted because it was measured against a noisy timing harness that showed nothing; at 6 calls it is 9% of the cycle's growth and worth reconsidering against the band, which can now see it. RAISING OR LOWERING A ROW IS A DECISION, not a maintenance chore. Append here with the interpreter and the harness invocation, as above. -- 2026-09-26 #546 -- the stages copy their state with `_state.copy_with` instead of `dataclasses.replace`, and every row moves DOWN. `dataclasses.replace` walks `fields()` and calls `__init__` on each copy, three frames on 3.11 and 3.12 and four from 3.13 where `copy_with` is one. One parse of the reference name makes 18 copies: six of the state, one each from tokenize, segment, classify, group, assign and post_rules, and twelve of single tokens, six each from classify and assign (`extract_delimited` returns the state unchanged when there is no delimiter, and `script_segment` returns early on ASCII input). That is the whole of the drop, 18 x 2 = 36 frames on 3.11 and 3.12 and 18 x 3 = 54 from 3.13 (recompute: wrap `copy_with` with a counter in the eight stage modules and parse the reference name). A field copy builds what `replace` builds for a dataclass that is decorated itself rather than inheriting the decoration, keeps the generated `__init__`, and has no `__post_init__` and no `init=False` field. `_copyable_fields` checks exactly those four, and `_COPY_FIELDS` runs it over `WorkToken`, `PendingAmbiguity` and `ParseState` at import, so a class that stops qualifying fails there and `copy_with` copies nothing else. `test_the_guard_refuses_a_class_a_field_copy_would_get_wrong` records, for each refused shape, what `replace` builds and what an unguarded copy would build instead. To mypy, `copy_with` is `from dataclasses import replace as copy_with`, which keeps the dataclass plugin's keyword and type checks at every call site; an assignment (`copy_with = dataclasses.replace`) would not, since the plugin keys on the callee's full name (measured: a misspelled field and a wrong-typed value both pass through the assignment and both fail through the import). Measured 2026-09-26 with `uv run python tools/perf/call_count.py --against e0f1a2f`, each row on its own interpreter, parse/facade: 3.11 406/443 → 370/407, 3.12 384/421 → 348/385, 3.13, 3.14 and 3.15 402/439 → 348/385. The rows drop by 40 and 58 rather than 36 and 54 because e0f1a2f already read 4 under every row, inside the band, and the new rows are set to what the harness reads now. `_LINK_BASELINE`'s 64-link clause reads 2587 → 2301 on 3.11. By stage (`--stages`, py3.11, ms per 1000 parses of the reference name): group 19.6 → 17.3, classify 13.4 → 9.8, assign 11.1 → 8.0, tokenize 8.1 → 6.6, post_rules 7.9 → 6.5, segment 2.6 → 1.7. BEHAVIOR IDENTICAL: the differential gate's report at all five baselines matches e0f1a2f's line for line apart from the path header. +- 2026-09-26 #546 (its four conditions superseded by 2026-09-29 #561, below; its rows superseded by 2026-10-06 #617, and its link pin by #601's re-pointed shape on 2026-10-04, which this file records no entry for) -- the stages copy their state with `_state.copy_with` instead of `dataclasses.replace`, and every row moves DOWN. `dataclasses.replace` walks `fields()` and calls `__init__` on each copy, three frames on 3.11 and 3.12 and four from 3.13 where `copy_with` is one. One parse of the reference name makes 18 copies: six of the state, one each from tokenize, segment, classify, group, assign and post_rules, and twelve of single tokens, six each from classify and assign (`extract_delimited` returns the state unchanged when there is no delimiter, and `script_segment` returns early on ASCII input). That is the whole of the drop, 18 x 2 = 36 frames on 3.11 and 3.12 and 18 x 3 = 54 from 3.13 (recompute: wrap `copy_with` with a counter in the eight stage modules and parse the reference name). A field copy builds what `replace` builds for a dataclass that is decorated itself rather than inheriting the decoration, keeps the generated `__init__`, and has no `__post_init__` and no `init=False` field. `_copyable_fields` checks exactly those four, and `_COPY_FIELDS` runs it over `WorkToken`, `PendingAmbiguity` and `ParseState` at import, so a class that stops qualifying fails there and `copy_with` copies nothing else. `test_the_guard_refuses_a_class_a_field_copy_would_get_wrong` records, for each refused shape, what `replace` builds and what an unguarded copy would build instead. To mypy, `copy_with` is `from dataclasses import replace as copy_with`, which keeps the dataclass plugin's keyword and type checks at every call site; an assignment (`copy_with = dataclasses.replace`) would not, since the plugin keys on the callee's full name (measured: a misspelled field and a wrong-typed value both pass through the assignment and both fail through the import). Measured 2026-09-26 with `uv run python tools/perf/call_count.py --against e0f1a2f`, each row on its own interpreter, parse/facade: 3.11 406/443 → 370/407, 3.12 384/421 → 348/385, 3.13, 3.14 and 3.15 402/439 → 348/385. The rows drop by 40 and 58 rather than 36 and 54 because e0f1a2f already read 4 under every row, inside the band, and the new rows are set to what the harness reads now. `_LINK_BASELINE`'s 64-link clause reads 2587 → 2301 on 3.11. By stage (`--stages`, py3.11, ms per 1000 parses of the reference name): group 19.6 → 17.3, classify 13.4 → 9.8, assign 11.1 → 8.0, tokenize 8.1 → 6.6, post_rules 7.9 → 6.5, segment 2.6 → 1.7. BEHAVIOR IDENTICAL: the differential gate's report at all five baselines matches e0f1a2f's line for line apart from the path header. - 2026-09-29 #561 -- SUPERSEDED IN PART: the 2026-09-26 bullet above, whose four conditions ("decorated itself", "keeps the generated `__init__`", no `__post_init__`, no `init=False` field) were not sufficient. `replace` builds through `obj.__class__(...)`, and a field copy skips every hook that call runs; four shapes met all four conditions and still parted: a validating `__new__`, a validating `__setattr__` on a non-frozen dataclass and a validating metaclass `__call__`, each giving `ValueError` from `replace` where an unguarded field copy built `value=-1`, and an `InitVar` with no default, which `replace` demands (a `ValueError` through 3.12, a `TypeError` from 3.13) and a field copy never sees. #561's first tightening (80554266) found only the first three, and by swapping the class's OWN `__dataclass_params__` for an inherited lookup it also let through an `__init__` borrowed from another dataclass under that dataclass's name, which the 2026-09-26 guard had refused; review caught both. So the guard is now a list of conditions a class must MEET (`_state._copy_refusals`, one reason string per unmet condition): decorated as a dataclass itself, frozen, an `__init__` generated for it (compiled from `` and named `.__init__`) that takes exactly its `init` fields, the default metaclass, no `__new__` above `object`, no `__post_init__`, no `init=False` field. It is a TRIPWIRE for a realistic edit to the three pipeline classes and says so, not a proof against any class: a forged `__qualname__` on a borrowed `__init__` in a class that is also decorated itself, or a `__class__` property, still passes, and `copy_with` copies nothing but `WorkToken`, `PendingAmbiguity` and `ParseState`, which meet every condition on 3.11 to 3.15. `_UNGUARDED_EFFECT` records for each row what `replace` builds, what an unguarded copy builds, and which conditions refuse it; `test_every_guard_condition_alone_refuses_a_recorded_shape` asserts that each condition is the only refusal of some row, so dropping one fails a test rather than contradicting this paragraph (checked by removing the `__new__` condition in a scratch copy: two failures). Call counts unchanged, the guard running at import only. -- 2026-10-10 #624 -- every row moves DOWN by 4: `parse` 308 → 304 on 3.11 and 289 → 285 on 3.12-3.15, `facade` 345 → 341 and 326 → 322, each measured on its own interpreter with `tools/perf/call_count.py` in isolated copies of 8e6e524a (#631's merge) and of the change, the import asserted (8e6e524a read exactly the recorded rows). `--modules` on 3.11: `_pieces.py` 51 → 49 and `_group.py` 25 → 23 -- group's leading-position scan, a generator asking `is_leading_title` per piece, gave way to `leading_titles`, which it already called, and `chain_lead`, which asks `is_prefix_piece` only of the titles. +- 2026-10-10 #624 (its rows superseded by 2026-10-10 #625) -- every row moves DOWN by 4: `parse` 308 → 304 on 3.11 and 289 → 285 on 3.12-3.15, `facade` 345 → 341 and 326 → 322, each measured on its own interpreter with `tools/perf/call_count.py` in isolated copies of 8e6e524a (#631's merge) and of the change, the import asserted (8e6e524a read exactly the recorded rows). `--modules` on 3.11: `_pieces.py` 51 → 49 and `_group.py` 25 → 23 -- group's leading-position scan, a generator asking `is_leading_title` per piece, gave way to `leading_titles`, which it already called, and `chain_lead`, which asks `is_prefix_piece` only of the titles. - 2026-10-10 #625 -- every row moves UP by 3: `parse` 304 → 307 on 3.11 and 285 → 288 on 3.12-3.15, `facade` 341 → 344 and 322 → 325, each measured on its own interpreter with `tools/perf/call_count.py` in isolated copies of 505d6dec (#632's merge) and of the change, the import asserted (505d6dec read exactly the recorded rows). `--modules` on 3.11: `_pieces.py` 49 → 52, `_group.py` 23 → 23. The reference name holds a particle run (`de la`), so it pays the merge's call and one more `merge_pieces` and `joined_tags`, the chain then merging `de la` with `Vega` where it had merged the three words at once; the first cut asked the title run and the lead in the merge and again in the read, +11, and group now asks the title run once and hands both on. Every name of three or more words read at the trailing site pays the call, +1 (`John Q. Smith` 202 → 203 on 3.11, warm); a name with a particle run pays the 3 (`Juan de la Vega` 230 → 233). -- 2026-10-06 #617 -- every row moves DOWN by 10: `parse` 374 → 364 on 3.11 and 353 → 343 on 3.12-3.15, `facade` 411 → 401 and 390 → 380, each measured on its own interpreter with `PYTHONPATH= pythonX.Y tools/perf/call_count.py` against 83f4e914 (every row had sat inside its band before, the 3.12-3.15 ones at +5 over a 348 baseline). `--modules` puts all ten in group and the piece layer (`_group.py` + `_pieces.py` 142 → 132, every other module unchanged): P3's connective joins moved out of `_group_segment` into `_pieces.join_connectives`, which asks `is_conj_piece`, `is_title_piece` and `is_prefix_piece` directly where the loops had asked through one-line closures over them, two frames per test becoming one. The link pin moves with it, 2,678 → 2,479 for the 64-link name on 3.11 (`_LINK_BASELINE`), and there for two reasons, counted per function in #617's review: the closures (`conj` 133 frames, `title` and `prefix` one each) and the single-letter test, which had joined each connective's text through a generator (65 frames, one per link, more than the pin's whole band), less the one `join_connectives` frame. A deliberate move, not drift: the change was a refactor (decisions/P3, 2026-10-06). The reference rows hold no comma; a comma part holding a connective now pays the shared loop (`John Smith, Mr. and Mrs.` 262 → 269 on 3.11, warm), and one holding none skips it and costs what it did (`Smith, John` 161), the review having found the first cut charging every family-comma parse two frames per word of the part. THEN 39 MORE on every row, in the branch's /simplify pass: group ran the rootname count (about five frames a piece, its only reader the carve-out) and the shared loop over every segment of three or more pieces, connective or not, and over the corpora and the case table under three orders 0 of the 3,153 group calls with no connective left free changed a piece. Group now skips both where its frozen-set walk, which already reads every piece's `conjunction` tag, finds no connective free, as `_comma._reading_pieces` does: `parse` 364 → 325 on 3.11 and 343 → 304 on 3.12-3.15, `facade` 401 → 362 and 380 → 341, `_group.py` + `_pieces.py` 132 → 93, measured on each interpreter as above; the link pin does not move, every link in it joining. Per name on 3.11, warm: the Vega name 436 → 386, `Smith, John Quincy Adams Bob` 326 → 297, `Smith, John` and `John Smith, Mr. and Mrs.` unchanged at 161 and 269. No output moved over the #617 grids (decisions/P3). -- 2026-10-07 -- every row moves DOWN by 8 more: `_group_segment`'s last three one-line predicate closures, `prefix` over `is_prefix_piece`, `suffix` over `is_suffix_piece` and `marker` over `_is_maiden_marker_piece`, are inlined as direct calls, finishing what #617 did for `title`, `conj` and `merge`. Each closure was a frame of its own ahead of the predicate's, and the chain's two inner scans ask `prefix` once per piece they pass, so the cost grows with particle pieces (frame-budget question 1 in AGENTS.md). Measured with `PYTHONSAFEPATH=1 PYTHONPATH= pythonX.Y tools/perf/call_count.py`, each row on its own interpreter, parse/facade against 0c54662b: 3.11 325/362 → 317/354, 3.12, 3.13, 3.14 and 3.15 304/341 → 296/333. `--modules` on 3.11 puts all eight in `_group.py` (34 → 26), every other module unchanged. The whole of the drop is the closures, counted per function on 0c54662b (a profile hook counting `call` events whose code is named `prefix`, `suffix` or `marker` in `_group.py`): the reference name asked `prefix` 7 times and `suffix` once, `Juan` + `de la Vega ` × 8 + `Smith` asked 40 and 9 and reads 937 → 888 on 3.11 (919 → 870 on 3.12-3.15), and the 64-link name asked `prefix` once, so `_LINK_BASELINE` moves 2,479 → 2,478 on 3.11 and gains rows for 3.12-3.15 at 2,459 (2,460 on 0c54662b), each measured on its own interpreter and the first the pin has had beyond 3.11; `marker` is reached only by P5's bound-join decline and the reference name never asks it. BEHAVIOR IDENTICAL: the full suite passes, and the differential gate's report at all five baselines matches 0c54662b's line for line apart from the path header. -- 2026-10-08 #620 -- no row moves: `parse` 308 on 3.11 and 289 on 3.12-3.15, `facade` 345 and 326, measured as the #614 bullet below on 4278693d (#621's merge) and on the change alike. The 3.12-3.15 pin read 290/327 until this date: #614's /simplify took one frame off those interpreters (289/326 on 4278693d, measured 2026-10-08) and the pin stayed, inside its band, so the #614 bullet's 290 is the pin's number, not that tree's. `--modules` on 3.11 unchanged. The merged copy and its per-word exclusion test are gone, and `_chain_units` asks `chain_run_end` per particle unit and `_weighed` per word behind one; the chain itself calls `chain_run_end` with the particle test inline where it had called `is_prefix_piece` per piece, which pays for the rest. decisions/S2, this date. -- 2026-10-07 #614 -- every row moves DOWN again, on top of the closure inlining above: `parse` 317 → 308 on 3.11 and 296 → 290 on 3.12-3.15, `facade` 354 → 345 and 333 → 327, each measured on its own interpreter with `PYTHONSAFEPATH=1 PYTHONPATH= pythonX.Y /tools/perf/call_count.py` against adf6da88 (#619's merge), the import asserted. `--modules` on 3.11: `_pieces.py` 59 → 51, `_group.py` 26 → 25, `_assign.py` 9 → 10, every other module unchanged (`_vocab.py` read 37 until #614's /simplify inlined the one roman test the counting view asks per word, 2026-10-07). S2's trailing run is read once between P3's joins and the particle chain, so the chain's stop and its re-ask, the reserve's two views and assign's own reading collapse into one, and the counting view it reads through is one call on every segment that reaches the read, and its unit walk (`_counted_as_one`) runs only on a segment holding a particle past its first piece; the link pin reads 2,478 → 2,482, inside its band, and stays. A segment of fewer than three pieces never reaches the read and costs what it did (`John Smith` 152, `Smith, John` 161, `John Smith, PhD` 206 on 3.11, warm). decisions/S2, this date. +- 2026-10-06 #617 (its rows and link pin superseded by 2026-10-07 #619) -- every row moves DOWN by 10: `parse` 374 → 364 on 3.11 and 353 → 343 on 3.12-3.15, `facade` 411 → 401 and 390 → 380, each measured on its own interpreter with `PYTHONPATH= pythonX.Y tools/perf/call_count.py` against 83f4e914 (every row had sat inside its band before, the 3.12-3.15 ones at +5 over a 348 baseline). `--modules` puts all ten in group and the piece layer (`_group.py` + `_pieces.py` 142 → 132, every other module unchanged): P3's connective joins moved out of `_group_segment` into `_pieces.join_connectives`, which asks `is_conj_piece`, `is_title_piece` and `is_prefix_piece` directly where the loops had asked through one-line closures over them, two frames per test becoming one. The link pin moves with it, 2,678 → 2,479 for the 64-link name on 3.11 (`_LINK_BASELINE`), and there for two reasons, counted per function in #617's review: the closures (`conj` 133 frames, `title` and `prefix` one each) and the single-letter test, which had joined each connective's text through a generator (65 frames, one per link, more than the pin's whole band), less the one `join_connectives` frame. A deliberate move, not drift: the change was a refactor (decisions/P3, 2026-10-06). The reference rows hold no comma; a comma part holding a connective now pays the shared loop (`John Smith, Mr. and Mrs.` 262 → 269 on 3.11, warm), and one holding none skips it and costs what it did (`Smith, John` 161), the review having found the first cut charging every family-comma parse two frames per word of the part. THEN 39 MORE on every row, in the branch's /simplify pass: group ran the rootname count (about five frames a piece, its only reader the carve-out) and the shared loop over every segment of three or more pieces, connective or not, and over the corpora and the case table under three orders 0 of the 3,153 group calls with no connective left free changed a piece. Group now skips both where its frozen-set walk, which already reads every piece's `conjunction` tag, finds no connective free, as `_comma._reading_pieces` does: `parse` 364 → 325 on 3.11 and 343 → 304 on 3.12-3.15, `facade` 401 → 362 and 380 → 341, `_group.py` + `_pieces.py` 132 → 93, measured on each interpreter as above; the link pin does not move, every link in it joining. Per name on 3.11, warm: the Vega name 436 → 386, `Smith, John Quincy Adams Bob` 326 → 297, `Smith, John` and `John Smith, Mr. and Mrs.` unchanged at 161 and 269. No output moved over the #617 grids (decisions/P3). +- 2026-10-07 #619 (its rows superseded by 2026-10-07 #614; #619 added 2026-10-22, the head having named no issue of its own: commit 343e268c landed through #619's merge, adf6da88) -- every row moves DOWN by 8 more: `_group_segment`'s last three one-line predicate closures, `prefix` over `is_prefix_piece`, `suffix` over `is_suffix_piece` and `marker` over `_is_maiden_marker_piece`, are inlined as direct calls, finishing what #617 did for `title`, `conj` and `merge`. Each closure was a frame of its own ahead of the predicate's, and the chain's two inner scans ask `prefix` once per piece they pass, so the cost grows with particle pieces (frame-budget question 1 in AGENTS.md). Measured with `PYTHONSAFEPATH=1 PYTHONPATH= pythonX.Y tools/perf/call_count.py`, each row on its own interpreter, parse/facade against 0c54662b: 3.11 325/362 → 317/354, 3.12, 3.13, 3.14 and 3.15 304/341 → 296/333. `--modules` on 3.11 puts all eight in `_group.py` (34 → 26), every other module unchanged. The whole of the drop is the closures, counted per function on 0c54662b (a profile hook counting `call` events whose code is named `prefix`, `suffix` or `marker` in `_group.py`): the reference name asked `prefix` 7 times and `suffix` once, `Juan` + `de la Vega ` × 8 + `Smith` asked 40 and 9 and reads 937 → 888 on 3.11 (919 → 870 on 3.12-3.15), and the 64-link name asked `prefix` once, so `_LINK_BASELINE` moves 2,479 → 2,478 on 3.11 and gains rows for 3.12-3.15 at 2,459 (2,460 on 0c54662b), each measured on its own interpreter and the first the pin has had beyond 3.11; `marker` is reached only by P5's bound-join decline and the reference name never asks it. BEHAVIOR IDENTICAL: the full suite passes, and the differential gate's report at all five baselines matches 0c54662b's line for line apart from the path header. +- 2026-10-08 #620 (its rows superseded by 2026-10-10 #624) -- no row moves: `parse` 308 on 3.11 and 289 on 3.12-3.15, `facade` 345 and 326, measured as the #614 bullet below on 4278693d (#621's merge) and on the change alike. The 3.12-3.15 pin read 290/327 until this date: #614's /simplify took one frame off those interpreters (289/326 on 4278693d, measured 2026-10-08) and the pin stayed, inside its band, so the #614 bullet's 290 is the pin's number, not that tree's. `--modules` on 3.11 unchanged. The merged copy and its per-word exclusion test are gone, and `_chain_units` asks `chain_run_end` per particle unit and `_weighed` per word behind one; the chain itself calls `chain_run_end` with the particle test inline where it had called `is_prefix_piece` per piece, which pays for the rest. decisions/S2, this date. +- 2026-10-07 #614 (its rows superseded by 2026-10-08 #620 on 3.12-3.15 and by 2026-10-10 #624 on 3.11) -- every row moves DOWN again, on top of the closure inlining above: `parse` 317 → 308 on 3.11 and 296 → 290 on 3.12-3.15, `facade` 354 → 345 and 333 → 327, each measured on its own interpreter with `PYTHONSAFEPATH=1 PYTHONPATH= pythonX.Y /tools/perf/call_count.py` against adf6da88 (#619's merge), the import asserted. `--modules` on 3.11: `_pieces.py` 59 → 51, `_group.py` 26 → 25, `_assign.py` 9 → 10, every other module unchanged (`_vocab.py` read 37 until #614's /simplify inlined the one roman test the counting view asks per word, 2026-10-07). S2's trailing run is read once between P3's joins and the particle chain, so the chain's stop and its re-ask, the reserve's two views and assign's own reading collapse into one, and the counting view it reads through is one call on every segment that reaches the read, and its unit walk (`_counted_as_one`) runs only on a segment holding a particle past its first piece; the link pin reads 2,478 → 2,482, inside its band, and stays. A segment of fewer than three pieces never reaches the read and costs what it did (`John Smith` 152, `Smith, John` 161, `John Smith, PhD` 206 on 3.11, warm). decisions/S2, this date. diff --git a/docs/design/decisions/review-agent-single-source.md b/docs/design/decisions/review-agent-single-source.md index ae2c7518..23400eb4 100644 --- a/docs/design/decisions/review-agent-single-source.md +++ b/docs/design/decisions/review-agent-single-source.md @@ -1,5 +1,14 @@ # review-agent-single-source — the review axes live in one file +## Standing + +- `.claude/agents/` and `.claude/commands/` are tracked, and the design-docs reviewer points at the review axes in `docs/design/AGENTS.md` rather than restating them; a duplicate that carries no independent content is deleted, not kept in sync — 2026-08-31 #473. +- A test pinning the agent's axis headings against the AGENTS.md list was declined: it would have caught one of the three drifts that earned this entry, the other two changing body prose under intact headings — 2026-08-31 #473. + +Open: none in this file. + +## Log + - 2026-08-31 (#473) — `.claude/agents/design-docs-reviewer.md` restated AGENTS.md's review axes and, being untracked, drifted three ways against the file it implements: axis 10 never arrived, axis 4's outward half never arrived, and and the corpus-population caution kept a wording AGENTS.md itself records as corrected. That third one carried a second defect neither copy could show: BOTH quoted a corpus measurement that decisions/P6 had already superseded, so the endorsed wording was stale too. AGENTS.md's caution is rewritten in this change to carry the lesson without a standing count. The drift was one-directional — every correction landed in the tracked, reviewed, diffable file and none reached the copy. `.claude/agents/` and `.claude/commands/` are now tracked, and the agent points at AGENTS.md rather than restating it. The general form: when a duplicate carries no independent content, delete it; a duplicate is only worth keeping when it says something its source does not. Declined: a test pinning the agent's axis headings against AGENTS.md's list, which is what #473 proposed. Heading comparison would have caught the missing axis 10 and NEITHER of the other two drifts — both changed body prose and left headings intact — so the test detects a third of the defect while licensing the arrangement that produced all of it. Same shape as the `STABLE_TAGS` twin flagged in `nameparser/_types.py`, and the same remedy applies there if it is ever revisited. diff --git a/tests/v2/decisions_without_standing.json b/tests/v2/decisions_without_standing.json index 02806807..e1a4eabd 100644 --- a/tests/v2/decisions_without_standing.json +++ b/tests/v2/decisions_without_standing.json @@ -1,11 +1,4 @@ { "_comment": "decisions files allowed to lack a ## Standing block: those created before the rule, C1's block landing on 2026-10-19 (C12 and vocabulary-model were created that day, before it). A file leaves the list the day it gains a block, or the day the index lists it under Reference lists; the list must stay inside STANDING_EXEMPT_ORIGINAL in tests/v2/design_docs.py, so it never holds a key it did not start with and a new file never joins it (tests/v2/test_design_docs.py::test_a_decisions_file_off_the_list_opens_with_standing; #762 decides the rest).", - "exempt": [ - "3-0-reevaluations", - "comma-suffix-arc", - "deviates-registry", - "differential-ledger", - "parse-cost", - "review-agent-single-source" - ] + "exempt": [] }