diff --git a/AGENTS.md b/AGENTS.md index e0f65aa8..979751c8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -194,7 +194,7 @@ The library has two layers: `nameparser/config/` (data) and `nameparser/parser.p **The default vocabulary is written per word, in TOML, and compiled.** `nameparser/vocab/.toml` holds one file per BCP 47 language tag, each `[[word]]` entry a `text` and its readings keyed by kind. What a file means, how an entry is built, the kinds and properties with the set each compiles into, exclusions, bearer counts and the steps for adding a word are in `nameparser/vocab/README.md`, the reference for anyone editing the TOML; `tests/v2/test_vocab.py` holds its kind, property and reserved-name tables to the names `build.py` accepts and the sets it compiles, so read it rather than restating it here. The parser never reads the TOML: `tools/vocab/build.py` compiles it into the committed `nameparser/config/_compiled.py`, frozenset literals one entry per line, and runs the build checks that replaced the modules' import-time asserts (its docstring numbers them). Each `config/` module re-exports its constants from there (`TITLES = _compiled.TITLES`) under its autodoc docstring, so the public names, the docs and the deprecated-alias bridge did not move. `tests/v2/test_vocab.py` fails when `_compiled.py` is stale, naming the refresh command, and gives every build check a mutation. The model, the phasing and what was declined: decisions/vocabulary-model. -The `config/` SETS are frozen since 2.2 (#293): there is no `.add()`/`.remove()` on any of them, so a default word list is changed at runtime by configuring an object — a private `Constants` for `HumanName`, a `Lexicon` for the 2.0 API — never by editing the constant, and the shipped default is changed by editing the TOML and rebuilding. `CONSTANTS`/`Constants` still hand out mutable `SetManager`s; the freeze is on the module set constants they copy from. **Neither dict was frozen, and `CAPITALIZATION_EXCEPTIONS` is not covered by anything else either.** `REGEXES` is a compiled-pattern table rather than vocabulary and was never in #293's scope; `CAPITALIZATION_EXCEPTIONS` is vocabulary-shaped and is a decided, in-scope exemption. So the split-default hazard the freeze closes is still live for it, measured on 2.2 and re-measured 2026-09-23 with the key below once `'PhD'` became the shipped value: `CAPITALIZATION_EXCEPTIONS['dphil'] = 'DPhil'` reaches a freshly built `Constants` and neither the cached `Lexicon.default()` nor the shared `CONSTANTS`. Same advice — configure the object (`constants.capitalization_exceptions[...]`, or `dataclasses.replace(lexicon, capitalization_exceptions=...)`). `tests/v2/test_contracts.py::test_every_vocabulary_constant_is_frozen` names both dicts as explicit exemptions rather than letting its `isinstance` filter drop them. +The `config/` SETS are frozen since 2.2 (#293): there is no `.add()`/`.remove()` on any of them, so a default word list is changed at runtime by configuring an object — a private `Constants` for `HumanName`, a `Lexicon` for the 2.0 API — never by editing the constant, and the shipped default is changed by editing the TOML and rebuilding. `CONSTANTS`/`Constants` still hand out mutable `SetManager`s; the freeze is on the module set constants they copy from. **Neither dict was frozen, and `CAPITALIZATION_EXCEPTIONS` is not covered by anything else either.** `REGEXES` is a compiled-pattern table rather than vocabulary and was never in #293's scope; `CAPITALIZATION_EXCEPTIONS` is vocabulary-shaped and is a decided, in-scope exemption. So the split-default hazard the freeze closes is still live for it, measured on 2.2 and re-measured 2026-10-25 with the key below, the earlier `dphil` example having become a shipped mask in #797: `CAPITALIZATION_EXCEPTIONS['dclinpsy'] = 'DClinPsy'` reaches a freshly built `Constants` and neither the cached `Lexicon.default()` nor the shared `CONSTANTS`. Same advice — configure the object (`constants.capitalization_exceptions[...]`, or `dataclasses.replace(lexicon, capitalization_exceptions=...)`). `tests/v2/test_contracts.py::test_every_vocabulary_constant_is_frozen` names both dicts as explicit exemptions rather than letting its `isinstance` filter drop them. - `titles.py` — `TITLES` (every title reading; prenominals), `GIVEN_NAME_TITLES` (the readings with `addresses = "given"`, e.g. "Sir", which treat the following name as given, not family — a subset by construction) and `AFTER_GIVEN_TITLES` (the salutation readings with `after_given = true`, read as the title where one ends the given part after a family comma, rules/H8, #835 — a subset by construction, with no v1 `Constants` attribute) - `suffixes.py` — `SUFFIX_ACRONYMS` (suffix readings with `acronym = true`, matched with every period removed, e.g. "M.D.") and `SUFFIX_WORDS` (the rest, e.g. "Jr."), disjoint because a word has one suffix kind; `SUFFIX_ACRONYMS_AMBIGUOUS` (`may_be_name`); and `GLUED_HONORIFICS` (#308, `attached`), the subset of `SUFFIX_WORDS` the peel may split off the END of a name token — a separate, harsher bar, since the glued position has no writer-drawn boundary to lean on, and each word refused it carries `excluded = { attached = … }` @@ -348,7 +348,7 @@ Don't use the bare `python3 -m doctest .rst` CLI (no `optionflags`) to che **Prefix-join uses value-based `list.index()`** in `join_on_conjunctions` — fragile when a token value repeats (e.g. a trailing title that's also a suffix acronym, or two `van`s); constrain such lookups to start at `i + 1`. See #100. -**Title vs suffix is positional for BARE words, and the leading period-abbreviation rule overrides even that** — a word matching `TITLES` at the front of a name becomes `title`; the same word matching `SUFFIX_ACRONYMS`/`SUFFIX_WORDS` at the end becomes `suffix` (never both, regardless of the word's real-world meaning). The `TITLES`/suffix overlap was audited in #296 (2026-08-23): the pure postnominals (`jr`, `junior`, `phd`, `do`, `se`) left `TITLES`, the v1-residue `dr`/`sra` left the suffix sets, and the ten words still in both (`ms`, `sr`, `lt`, `ra`, `vc`, and the ranks `cpl`, `cpo`, `cpt`, `csm`, `sgm`; `md` was a twelfth until #853 took its title reading out, a leading `Md` being the given name Muhammad) are deliberate duals that position decides. External test sources (old issue gists, etc.) sometimes assert `suffix` for a leading professional abbreviation like `RA`/`PD`/`Dipl.-Ing.` — that's the source data being wrong, not a parser bug. Verify position before "fixing" it. Two qualifications the older "purely positional" wording papered over, the first measured 2026-08-01 and the second answered 2026-09-08 (#316): a PERIOD-marked leading word is claimed by the shape rule before any vocabulary is read (`"Esq. Smith"` → `title`, though `esq` is suffix-only), and a PERIOD-marked trailing word is claimed by VOCABULARY — first the suffix sets, which the peel reads before anything else (`"John Smith Esq."` → `suffix`), then the titles (`"John Smith Prof."` → `title='Prof.'`, `rules/H5`) — while an unlisted abbreviation there stays a name part (`"John Smith Xyz."` → `family='Xyz.'`) unless it is written as two or more period-separated chunks, which since 2.4 IS a trailing shape rule (`Policy.unlisted_dotted_suffixes`, default on: `"John Smith X.Y.Z."` → `suffix='X.Y.Z.'`, read by the same words-to-spare count a bare ambiguous acronym takes, `rules/S3`), except paired initials in a script with capitals, which stay a name word (`"Ram Bahadur B.K."` → `family='B.K.'`, #864). The SINGLE trailing period is still not one, deliberately — it is the abbreviation shape any word can wear. The bare word is where "positional" still holds whole -- outside a part past the second comma, which holds no name word, so C2 reads a bare title word there as a title since #603 (`Eric H. Holder, Jr., Attorney General`), and outside a listed salutation ending the given part after a family comma, which rules/H8 reads as the title since #835 (`Doe, Jane Mrs`), the comma having marked the surname's place -- and the reason it must: `TITLES` holds words that are in no suffix set — 787 of them on 2026-10-11 (746 before #606's salutations), measured with `L = Parser().lexicon; len(L.titles - L.suffix_acronyms - L.suffix_words)`, a figure that grows with the vocabulary and never shrinks the argument — and many are ordinary surnames (`king`, `bishop`, `prince`, `pope`, `judge`, `sheriff`, `baron`, `master`, ...), so a vocabulary-first rule over BARE trailing words would read `"Mary Jane King"` as `title='King'`, `family='Jane'`. The period is what separates the safe case from that one, and it separates it by being a WRITING convention rather than by making the collision go away: `"Mary Jane King"` keeps `family='King'` while `"Mary Jane King."` reads `title='King.'`, `family='Jane'`, a cost accepted under the input-is-a-name premise rather than one the rule prevents (the trailing-position bullet in decisions/vocabulary-exclusions, measured 2026-09-09). Since #316 that sentence describes the shipped rule rather than an aspiration. +**Title vs suffix is positional for BARE words, and the leading period-abbreviation rule overrides even that** — a word matching `TITLES` at the front of a name becomes `title`; the same word matching `SUFFIX_ACRONYMS`/`SUFFIX_WORDS` at the end becomes `suffix` (never both, regardless of the word's real-world meaning). The `TITLES`/suffix overlap was audited in #296 (2026-08-23): the pure postnominals (`jr`, `junior`, `phd`, `do`, `se`) left `TITLES`, the v1-residue `dr`/`sra` left the suffix sets, and the ten words still in both (`ms`, `sr`, `lt`, `ra`, `vc`, and the ranks `cpl`, `cpo`, `cpt`, `csm`, `sgm`; `md` and `sa` were duals too until #853 took `md`'s title reading out, a leading `Md` being the given name Muhammad, and #857 took `sa` out of the vocabulary) are deliberate duals that position decides. External test sources (old issue gists, etc.) sometimes assert `suffix` for a leading professional abbreviation like `RA`/`PD`/`Dipl.-Ing.` — that's the source data being wrong, not a parser bug. Verify position before "fixing" it. Two qualifications the older "purely positional" wording papered over, the first measured 2026-08-01 and the second answered 2026-09-08 (#316): a PERIOD-marked leading word is claimed by the shape rule before any vocabulary is read (`"Esq. Smith"` → `title`, though `esq` is suffix-only), and a PERIOD-marked trailing word is claimed by VOCABULARY — first the suffix sets, which the peel reads before anything else (`"John Smith Esq."` → `suffix`), then the titles (`"John Smith Prof."` → `title='Prof.'`, `rules/H5`) — while an unlisted abbreviation there stays a name part (`"John Smith Xyz."` → `family='Xyz.'`) unless it is written as two or more period-separated chunks, which since 2.4 IS a trailing shape rule (`Policy.unlisted_dotted_suffixes`, default on: `"John Smith X.Y.Z."` → `suffix='X.Y.Z.'`, read by the same words-to-spare count a bare ambiguous acronym takes, `rules/S3`), except paired initials in a script with capitals, which stay a name word (`"Ram Bahadur B.K."` → `family='B.K.'`, #864). The SINGLE trailing period is still not one, deliberately — it is the abbreviation shape any word can wear. The bare word is where "positional" still holds whole -- outside a part past the second comma, which holds no name word, so C2 reads a bare title word there as a title since #603 (`Eric H. Holder, Jr., Attorney General`), and outside a listed salutation ending the given part after a family comma, which rules/H8 reads as the title since #835 (`Doe, Jane Mrs`), the comma having marked the surname's place -- and the reason it must: `TITLES` holds words that are in no suffix set — 787 of them on 2026-10-11 (746 before #606's salutations), measured with `L = Parser().lexicon; len(L.titles - L.suffix_acronyms - L.suffix_words)`, a figure that grows with the vocabulary and never shrinks the argument — and many are ordinary surnames (`king`, `bishop`, `prince`, `pope`, `judge`, `sheriff`, `baron`, `master`, ...), so a vocabulary-first rule over BARE trailing words would read `"Mary Jane King"` as `title='King'`, `family='Jane'`. The period is what separates the safe case from that one, and it separates it by being a WRITING convention rather than by making the collision go away: `"Mary Jane King"` keeps `family='King'` while `"Mary Jane King."` reads `title='King.'`, `family='Jane'`, a cost accepted under the input-is-a-name premise rather than one the rule prevents (the trailing-position bullet in decisions/vocabulary-exclusions, measured 2026-09-09). Since #316 that sentence describes the shipped rule rather than an aspiration. ### Tests (`tests/`) diff --git a/CITATION.cff b/CITATION.cff index 88a01f09..f9690c93 100644 --- a/CITATION.cff +++ b/CITATION.cff @@ -9,4 +9,4 @@ authors: repository-code: "https://github.com/derek73/python-nameparser" url: "https://github.com/derek73/python-nameparser" license: LGPL-2.1-or-later -version: "2.5.0" +version: "2.6.0" diff --git a/README.rst b/README.rst index 237b5810..4ba93790 100644 --- a/README.rst +++ b/README.rst @@ -7,12 +7,12 @@ nameparser parses human names into seven fields — title, given, middle, family, suffix, nickname, maiden. Results are immutable, configuration is composable, and locale packs are opt-in. -📣 **nameparser 2.5 is out.** It reads junior and senior written in 45 -languages (``Antonio Pavlović mlađi``), keeps a title written before a -family comma out of the surname (``Dr. Smith, John``), reads Chinese and -Japanese credential and office words (``田中社長``), and gives a lone first -name followed by V, I or X its last name back (``Malcolm X``). The `release -log `__ has +📣 **nameparser 2.6 is out.** It keeps surnames that earlier releases +read as credentials (``Luo Si``, ``Maria de Sa``, ``Ram Bahadur B.K.``), +reads ``Wm`` and ``Md`` as the given names they abbreviate, recognizes more credentials after a comma +(``John Smith, MBChB``), and adds an opt-in reading for names with two +family names (``Gabriel García Márquez``). The `release log +`__ has the details. Chinese, Japanese and Korean names written in their own scripts are read diff --git a/docs/concepts.rst b/docs/concepts.rst index 962eaeb6..01ca5629 100644 --- a/docs/concepts.rst +++ b/docs/concepts.rst @@ -71,7 +71,9 @@ Whatever the vocabulary layer has not claimed is left to a positional layer, which assigns purely by where a word sits: the first unclaimed word is the given name, the last is the family name, and anything between them is the middle name. ``name_order``, an explicit comma -and ``script_orders`` change what "first" and "last" mean here. +and ``script_orders`` change what "first" and "last" mean here, as +does the opt-in ``two_family_names``, which takes the last two name +words as the family (:ref:`two-family-names`). ``script_orders`` covers the East Asian writing that settles its own order: a name wholly in Han or hangul, or a Japanese name in kanji and kana — any of them, hiragana alone included — unless it is katakana diff --git a/docs/customize.rst b/docs/customize.rst index 99f977f3..31d3346b 100644 --- a/docs/customize.rst +++ b/docs/customize.rst @@ -843,8 +843,8 @@ the family under the switch, and Argentine names, which carry one family name, are mostly that shape. Portuguese and Brazilian names carry two family names too, the mother's first, but the people who write them often give the last one alone as their family name: among -the UK company register's Portuguese filers the switch reads more -names wrong than it mends. The parser reports nothing for the names the switch +the UK company register's 5,333 Portuguese filers the switch breaks +1,481 names and mends 1,221 (measured 2026-10-24). The parser reports nothing for the names the switch moves: the shape it reads is the same either way. .. _east-asian-defaults: diff --git a/docs/design/decisions/C13-credential-opening-the-part-after-the-comma.md b/docs/design/decisions/C13-credential-opening-the-part-after-the-comma.md index 7b7cf3b0..46753b75 100644 --- a/docs/design/decisions/C13-credential-opening-the-part-after-the-comma.md +++ b/docs/design/decisions/C13-credential-opening-the-part-after-the-comma.md @@ -13,7 +13,7 @@ Open: none in this file. ## Log -- 2026-10-04 (Derek), #603 (decided at group's head since #613, decisions/C1; its `John Smith, MD Jones` and `MD - PhD - FACS` readings superseded by #853, decisions/indic-honorifics, `MD` being no title there) — A CREDENTIAL OPENING THE PART AFTER THE COMMA MAKES IT THE POSTNOMINAL PART, WHATEVER FOLLOWS; AND A TITLE WORD PAST THE SECOND COMMA IS A TITLE. One word that is neither suffix nor title vocabulary used to flip the whole comma reading: `John Smith, PhD Jones` read family 'John Smith', given 'PhD', where `John Smith, MD FACS` and `Eric H. Holder, Jr. Attorney General` were already read as a name and its postnominals. Now the first suffix word of the part, with only titles in front of it, opens the part when it starts S2's run (`_pieces.starts_a_credential_run`, #602's own predicate, so the two runs cannot disagree about what starts one), and every word after it reads as that run reads: a title word as a title, any other as a suffix. The mechanism was the one #296 built for a part holding no name word, extended rather than added beside; where this entry had segment deciding it, #613 (decisions/C1) decides it once at group's head. Every word the opened part takes that would otherwise have made it the given part is reported (`suffix-or-name`), as #602's run reports what it absorbs. +- 2026-10-04 (Derek), #603 (decided at group's head since #613, decisions/C1; its `John Smith, MD Jones`, `MD - PhD - FACS` and `Smith, MD PhD Ma` readings and `md`'s dual status superseded by #853, decisions/indic-honorifics, `MD` being no title there) — A CREDENTIAL OPENING THE PART AFTER THE COMMA MAKES IT THE POSTNOMINAL PART, WHATEVER FOLLOWS; AND A TITLE WORD PAST THE SECOND COMMA IS A TITLE. One word that is neither suffix nor title vocabulary used to flip the whole comma reading: `John Smith, PhD Jones` read family 'John Smith', given 'PhD', where `John Smith, MD FACS` and `Eric H. Holder, Jr. Attorney General` were already read as a name and its postnominals. Now the first suffix word of the part, with only titles in front of it, opens the part when it starts S2's run (`_pieces.starts_a_credential_run`, #602's own predicate, so the two runs cannot disagree about what starts one), and every word after it reads as that run reads: a title word as a title, any other as a suffix. The mechanism was the one #296 built for a part holding no name word, extended rather than added beside; where this entry had segment deciding it, #613 (decisions/C1) decides it once at group's head. Every word the opened part takes that would otherwise have made it the given part is reported (`suffix-or-name`), as #602's run reports what it absorbs. THE ONE-WORD PATH, DECIDED (Derek): the whole part is suffixes. `Doe, PhD Jones` reads family 'Doe', suffix 'PhD Jones', no given name, as `Smith, PhD` already reads. DECLINED: reading the credential as a suffix and the next name word as the given name (`Smith, PhD Jones` → given 'Jones'), proposed by analogy with a title at the head of the given part. Asked whether that is a real format, it is not -- nobody writes `Family, Credential Given` on purpose -- and the one realistic input it served, a misplaced generation (`Smith, Jr. John` for `Smith Jr., John`), is garbage in, garbage out. One reading for both comma paths was the deciding ground: the one-word path disagreeing with the full-name path is the shape #429, #430 and #432 each found. Accepted cost: `Smith, Jr. John` loses 'John' to the suffix, where 2.3 read given 'John' with 'Jr.' misfiled as a title. A TITLE/SUFFIX DUAL OPENS NOTHING (Derek). A word in both vocabularies (`md`, `ms`, `sr`, the ranks `lt`, `cpl` and the rest) is the listing form's title in front of a given name, `Smith, Ms Jane`, and a credential behind it opens nothing either (`Smith, Ms PhD Jones` keeps given 'PhD', as `Smith, MD PhD Ma` keeps given 'Ma' under #544). DECLINED after building it: a dual opening the part behind two or more words before the comma. Derek first chose it for `John Smith, MD Jones`; the reviews found what the count cannot see -- a surname of two words (`García Márquez, Ms Gabriela`, `Smith Jones, Sr. Maria`), a title in front of a one-word surname (`Dr. Smith, Ms Jane`, the count taking the title as a word), and spaced initials (`García Márquez, Ms G. J.`) each lost the given name -- and with the word explained (a dual is a word in both lists, decided by position), Derek chose that duals never open. So `John Smith, MD Jones` and the `MD - PhD - FACS` delimiter rows keep their 2.3 reading. An ambiguous-class member opens nothing either (`John Smith, Ma Jones` keeps given 'Ma'), nor does a suffix word S2's run start refuses (`John Smith, vd Jones`). STRICT MODE: the veto refuses an initial-shaped word the suffix READING, not the run. `Smith, PSM I.` under `lenient_comma_suffixes=False` reads suffix 'PSM I.' (2.3: given 'PSM', suffix 'I.'), reporting the absorbed 'I.', which matches #602's no-comma run (`John Smith PhD V.` reports; its given-part run does not, a disagreement #602 left and this does not touch). diff --git a/docs/design/decisions/S7-company-of-an-unambiguous-credential.md b/docs/design/decisions/S7-company-of-an-unambiguous-credential.md index 20c30f6e..8b84246c 100644 --- a/docs/design/decisions/S7-company-of-an-unambiguous-credential.md +++ b/docs/design/decisions/S7-company-of-an-unambiguous-credential.md @@ -20,7 +20,7 @@ Open: none in this file. ## Log -- 2026-09-27 (Derek), #544 (the `doe, jane v phd do` reading in its (7) and its 2026-09-28 addendum superseded by #721, decisions/S10: the one-case `v` is an initial and ends the run, so the name reads middle 'v', suffix 'phd do', the company still taking 'do') (its (6) and the addendum's `Jane Doe, PhD, Jr nee van Ma` superseded by #601, decisions/M2, a marker behind a credential or past a comma being an ordinary word; its ACCEPTED LIMITS' `John Smith PhD Prof. Ma`, `John Smith PhD Do Do` and `Smith, PhD Do Ma` superseded by the credential's run, #602, decisions/S8, and #603, decisions/C13, which read each as suffixes; (d)'s `John Smith PhD V Ma` superseded the same way, suffix 'PhD V Ma'; (c)'s `John Om Ma` superseded by the 2026-10-21 marking of `om` ambiguous, decisions/suffix-acronym-collisions, so it reads family 'Ma') — A CREDENTIAL RUN KEEPS ITS AMBIGUOUS MEMBERS: C1'S NAME-WORD COUNT READS A RUN (rules/C4 since #728) AS IT READS ONE WORD, AN UNAMBIGUOUS CREDENTIAL IN FRONT OF A MEMBER SPEAKS FOR IT AT EVERY TRAILING SLOT, AND A LISTED MEMBER'S CHUNKED DOTTED SPELLING PASSES THE PERIOD GATE. Three readers never looked past the member itself: C1 flipped a part to the suffix comma by the name-word count only when it was ONE token, and the trailing readers — the no-comma peel, the comma part read wholly as credentials, the given part's trailing slot — met the member first, took its name lean and stopped, so the credential in front of it was never reached. #540 marked `meng` and `lac`, whose conventional spellings lean name, which turned the one-case-row cost decisions/S6's 2026-09-15 item (ii) accepted into the ordinary spelling of two degrees. DECIDED, ten forks, (9) and (10) on 2026-09-28. (1) THE WHOLE RUN AT C1: two or more words after the comma, every one suffix vocabulary or a class candidate, at least one a candidate, behind two or more name words, is the credential run — all-member runs (`John Smith, Ed Ma`) and a leading member (`John Smith, MEng PhD`) included; offered and declined were an anchored-only rule and the issue's own "last word ambiguous" wording, which contradicted its `Ed Ma` consequence. (2) ALL TRAILING PATHS, ANCHORED IN FRONT: a listed member with an unambiguous credential in front of it, through other members, reads as the credential on the no-comma peel, in the comma part read wholly as credentials and at the given slot; a credential BEHIND it anchors nothing (`Wang Ma PhD` keeps family 'Ma'). (3) THE CHUNKED DOTTED GATE: a listed member written in two or more period-closed letter chunks (`M.Eng.`, `L.Ac.`) passes as `M.A.` does; one trailing period (`Ma.`, `Ed.`) still does not. (4) SINGLE-LETTER ROMAN NUMERALS neither anchor nor count toward a C1 run, S3's single-character retirement being the precedent; they peel as before, and the anchor's half of the exclusion is a matter of their shape, boundary (d) below. (5) NO NEW REPORT WHERE THE WRITING SETTLED IT: a run whose every member is LISTED and leans credential — S2's lean, capitals in a mixed-case name — is not flipped, the family-comma path already reading it as the credential run with the fields and the report it always had (`John Smith, PhD MA`) — narrowed 2026-10-01 by decisions/C4's #562 entry: a part holding two particles side by side is read by the count instead, and reports; the alternative, flipping such a run and suppressing only the flip's report, removes reports the family-comma path makes today, `John Smith, MA Jr` losing the one it makes for 'MA'. Because the lean is the listed set's, a by-shape member in capitals never settles a run: `John Smith, X.Y.Z. MA` flips to suffix 'X.Y.Z. MA' and reports. (6) THE ANCHOR DOES NOT REACH ACROSS A MAIDEN CLAUSE: `Jane Doe Jr. nee Smith Ma` keeps maiden 'Smith Ma' where `Jane Doe Jr. Ma` reads suffix 'Jr. Ma' — rules/M2's Accepted boundary, and the M2 bullet of this date. (7) AN ANCHOR OVERRIDES P6 AT THE GIVEN SLOT: `doe, jane v phd do` reads suffix 'v phd do' where it read family 'do doe'; the bullet below amends #531's pairing. (8) THE DUAL EXCLUSION IS SCOPED TO THE GIVEN PART'S LEADING TITLE RUN, NOT THE C1 RUN: after a full name the dual counts as the suffix vocabulary it is, as the legacy disjunct already counted it (`John Smith, MS MA`, `John Smith, MD`), and that is what reads the issue's headline `Jane Doe, MS LAc` as suffix 'MS LAc'. (9) A MEMBER THE COMPANY DECIDES REPORTS WHERE IT WAS DECIDED, in a family comma's part read wholly as credentials as at the peel: each member whose own writing declined and which the credential in front made the credential reports `suffix-or-name` at that reading. `Smith, PhD Ma`, `Smith, PhD MEng` and `Smith, Ph. D. MEng` report the member; `Smith, Dr. PhD LAc` and `John Quincy Smith, Prof. PhD LAc` report 'LAc'; `john smith, v phd ma` reports 'ma'; `John Smith, PhD Ma Prof.`, which C1 does not flip for its trailing title, reports 'Ma' where its twin `John Smith, PhD Ma` reports C1's flip once over the part. A member its own capitals made the credential stays silent (`Smith, PhD MA`), and the part's first piece, which nothing stands in front of, is the first-slot report's word and never this one's. (10) A DUAL IN THE GIVEN PART'S LEADING TITLE RUN SILENCES THE WHOLE PART: refinement (b) below reads such a dual as a title that speaks for nothing, and in a part whose leading title run holds one no credential behind the dual speaks for a member of the ambiguous class either, so the company never reads such a part wholly as credentials: the titles give it its reading, the dual a title and the next word the given name, and behind that given name the given slot reads as it reads any (`Smith, MD PhD Jr Ma` reads suffix 'Jr Ma', as `Smith, MD Jane Jr Ma` does). A part with no member to speak for, or only members whose capitals decide them, reads as it did: `Smith, MD PhD` suffix 'MD PhD', `Smith, Ms MD MA` suffix 'Ms MD MA'. `Smith, Ms MD Ma`, `Smith, MD MS Ma` and `SMITH, MD MS BA` read title 'Ms MD' / 'MD MS', given 'Ma' / 'BA'; `Smith, MD PhD Ma` reads title 'MD', given 'PhD', middle 'Ma', the given slot reporting 'Ma'; e10e83b4 read each of them so. The run is the walk's leading title run, so a dual behind a plain title stands in it too (`Smith, Prof. MD Ma` and `smith, prof. md ma` read title 'Prof. MD', given 'Ma'; `Smith, Dr. MD PhD Ma` keeps given 'PhD'), while a plain title alone silences nothing (`Smith, Dr. PhD LAc` reads title 'Dr.', suffix 'PhD LAc'). Behind a real given name the company reads as ever (`Doe, Jane MD Ma` and `Doe, Jane Ms Ma` read suffix 'MD Ma' and 'Ms Ma'), and past the title run a dual speaks like any suffix word (`Smith, PhD Ms Ma` reads suffix 'PhD Ms Ma'); `Smith, Ms Ma`, `Nguyen, Sr Ba` and `Smith, MD Do` are unchanged. +- 2026-09-27 (Derek), #544 (the `doe, jane v phd do` reading in its (7) and its 2026-09-28 addendum superseded by #721, decisions/S10: the one-case `v` is an initial and ends the run, so the name reads middle 'v', suffix 'phd do', the company still taking 'do') (its (10)'s `md` examples superseded by #853, decisions/indic-honorifics, `md` being no title: `Smith, MD PhD Ma` reads suffix 'MD PhD Ma') (its (6) and the addendum's `Jane Doe, PhD, Jr nee van Ma` superseded by #601, decisions/M2, a marker behind a credential or past a comma being an ordinary word; its ACCEPTED LIMITS' `John Smith PhD Prof. Ma`, `John Smith PhD Do Do` and `Smith, PhD Do Ma` superseded by the credential's run, #602, decisions/S8, and #603, decisions/C13, which read each as suffixes; (d)'s `John Smith PhD V Ma` superseded the same way, suffix 'PhD V Ma'; (c)'s `John Om Ma` superseded by the 2026-10-21 marking of `om` ambiguous, decisions/suffix-acronym-collisions, so it reads family 'Ma') — A CREDENTIAL RUN KEEPS ITS AMBIGUOUS MEMBERS: C1'S NAME-WORD COUNT READS A RUN (rules/C4 since #728) AS IT READS ONE WORD, AN UNAMBIGUOUS CREDENTIAL IN FRONT OF A MEMBER SPEAKS FOR IT AT EVERY TRAILING SLOT, AND A LISTED MEMBER'S CHUNKED DOTTED SPELLING PASSES THE PERIOD GATE. Three readers never looked past the member itself: C1 flipped a part to the suffix comma by the name-word count only when it was ONE token, and the trailing readers — the no-comma peel, the comma part read wholly as credentials, the given part's trailing slot — met the member first, took its name lean and stopped, so the credential in front of it was never reached. #540 marked `meng` and `lac`, whose conventional spellings lean name, which turned the one-case-row cost decisions/S6's 2026-09-15 item (ii) accepted into the ordinary spelling of two degrees. DECIDED, ten forks, (9) and (10) on 2026-09-28. (1) THE WHOLE RUN AT C1: two or more words after the comma, every one suffix vocabulary or a class candidate, at least one a candidate, behind two or more name words, is the credential run — all-member runs (`John Smith, Ed Ma`) and a leading member (`John Smith, MEng PhD`) included; offered and declined were an anchored-only rule and the issue's own "last word ambiguous" wording, which contradicted its `Ed Ma` consequence. (2) ALL TRAILING PATHS, ANCHORED IN FRONT: a listed member with an unambiguous credential in front of it, through other members, reads as the credential on the no-comma peel, in the comma part read wholly as credentials and at the given slot; a credential BEHIND it anchors nothing (`Wang Ma PhD` keeps family 'Ma'). (3) THE CHUNKED DOTTED GATE: a listed member written in two or more period-closed letter chunks (`M.Eng.`, `L.Ac.`) passes as `M.A.` does; one trailing period (`Ma.`, `Ed.`) still does not. (4) SINGLE-LETTER ROMAN NUMERALS neither anchor nor count toward a C1 run, S3's single-character retirement being the precedent; they peel as before, and the anchor's half of the exclusion is a matter of their shape, boundary (d) below. (5) NO NEW REPORT WHERE THE WRITING SETTLED IT: a run whose every member is LISTED and leans credential — S2's lean, capitals in a mixed-case name — is not flipped, the family-comma path already reading it as the credential run with the fields and the report it always had (`John Smith, PhD MA`) — narrowed 2026-10-01 by decisions/C4's #562 entry: a part holding two particles side by side is read by the count instead, and reports; the alternative, flipping such a run and suppressing only the flip's report, removes reports the family-comma path makes today, `John Smith, MA Jr` losing the one it makes for 'MA'. Because the lean is the listed set's, a by-shape member in capitals never settles a run: `John Smith, X.Y.Z. MA` flips to suffix 'X.Y.Z. MA' and reports. (6) THE ANCHOR DOES NOT REACH ACROSS A MAIDEN CLAUSE: `Jane Doe Jr. nee Smith Ma` keeps maiden 'Smith Ma' where `Jane Doe Jr. Ma` reads suffix 'Jr. Ma' — rules/M2's Accepted boundary, and the M2 bullet of this date. (7) AN ANCHOR OVERRIDES P6 AT THE GIVEN SLOT: `doe, jane v phd do` reads suffix 'v phd do' where it read family 'do doe'; the bullet below amends #531's pairing. (8) THE DUAL EXCLUSION IS SCOPED TO THE GIVEN PART'S LEADING TITLE RUN, NOT THE C1 RUN: after a full name the dual counts as the suffix vocabulary it is, as the legacy disjunct already counted it (`John Smith, MS MA`, `John Smith, MD`), and that is what reads the issue's headline `Jane Doe, MS LAc` as suffix 'MS LAc'. (9) A MEMBER THE COMPANY DECIDES REPORTS WHERE IT WAS DECIDED, in a family comma's part read wholly as credentials as at the peel: each member whose own writing declined and which the credential in front made the credential reports `suffix-or-name` at that reading. `Smith, PhD Ma`, `Smith, PhD MEng` and `Smith, Ph. D. MEng` report the member; `Smith, Dr. PhD LAc` and `John Quincy Smith, Prof. PhD LAc` report 'LAc'; `john smith, v phd ma` reports 'ma'; `John Smith, PhD Ma Prof.`, which C1 does not flip for its trailing title, reports 'Ma' where its twin `John Smith, PhD Ma` reports C1's flip once over the part. A member its own capitals made the credential stays silent (`Smith, PhD MA`), and the part's first piece, which nothing stands in front of, is the first-slot report's word and never this one's. (10) A DUAL IN THE GIVEN PART'S LEADING TITLE RUN SILENCES THE WHOLE PART: refinement (b) below reads such a dual as a title that speaks for nothing, and in a part whose leading title run holds one no credential behind the dual speaks for a member of the ambiguous class either, so the company never reads such a part wholly as credentials: the titles give it its reading, the dual a title and the next word the given name, and behind that given name the given slot reads as it reads any (`Smith, MD PhD Jr Ma` reads suffix 'Jr Ma', as `Smith, MD Jane Jr Ma` does). A part with no member to speak for, or only members whose capitals decide them, reads as it did: `Smith, MD PhD` suffix 'MD PhD', `Smith, Ms MD MA` suffix 'Ms MD MA'. `Smith, Ms MD Ma`, `Smith, MD MS Ma` and `SMITH, MD MS BA` read title 'Ms MD' / 'MD MS', given 'Ma' / 'BA'; `Smith, MD PhD Ma` reads title 'MD', given 'PhD', middle 'Ma', the given slot reporting 'Ma'; e10e83b4 read each of them so. The run is the walk's leading title run, so a dual behind a plain title stands in it too (`Smith, Prof. MD Ma` and `smith, prof. md ma` read title 'Prof. MD', given 'Ma'; `Smith, Dr. MD PhD Ma` keeps given 'PhD'), while a plain title alone silences nothing (`Smith, Dr. PhD LAc` reads title 'Dr.', suffix 'PhD LAc'). Behind a real given name the company reads as ever (`Doe, Jane MD Ma` and `Doe, Jane Ms Ma` read suffix 'MD Ma' and 'Ms Ma'), and past the title run a dual speaks like any suffix word (`Smith, PhD Ms Ma` reads suffix 'PhD Ms Ma'); `Smith, Ms Ma`, `Nguyen, Sr Ba` and `Smith, MD Do` are unchanged. WHAT MAY ANCHOR, four boundaries of the company clause, each a rule. (a) A CONNECTIVE (P3) anchors nothing and ends the run: the generational `i` is also Catalan's conjunction, and `Jane Doe nee Puig i Ma` keeps its link, maiden 'Puig i Ma'. (b) A TITLE/SUFFIX DUAL IN THE GIVEN PART'S LEADING TITLE RUN after a one-word family comma is a title there and anchors nothing — and, by (10), no credential behind it speaks for a member in that part — so the given slot's pass starts past the leading title run: `Smith, Ms Ma`, `Nguyen, Sr Ba` and `Smith, MD Do` keep their given names, and `Smith, MD MA Ma` keeps middle 'Ma', no unambiguous credential standing in its run once the title is set aside. Everywhere else a dual anchors like any suffix word (`John Smith MD MEng`, and (8)). (c) THE WALK'S OWN LEADING PIECE NEVER ANCHORS. In a name with no comma before the run, the leading piece is the name word H4's carve-out keeps whatever its vocabulary, and letting it anchor spends that name: in `Om Ma` and `PhD Ma` 'Ma' stays a name word under every name order (family 'Ma' in the default order, given 'Ma' under both family-first orders), where an anchoring `PhD` would leave given 'PhD', suffix 'Ma' and no family at all. One name word in front frees the same word — `John Om Ma` reads suffix 'Om Ma'. A family comma's given part has no such position, its first piece being read by the comma part's own credential reading, so `Smith, PhD MEng` reads family 'Smith', suffix 'PhD MEng'. (d) A SINGLE-LETTER ROMAN NUMERAL, IN ANY CASE (`V`, `v`, `I.`), anchors nothing for its SHAPE — one letter is how a middle initial is written, and S3 retired single-character vocabulary matches for the same reason (`John Smith PhD V Ma` keeps family 'Ma') — not because a generation is no credential. The rule is stated by that shape and the code tests exactly it (`_vocab.is_single_letter_numeral`); "initial-shaped" was the wrong word for it, `is_initial_shaped` answering False for a lower-case `v` that the exclusion covers too. And the exclusion is not about generations: a multi-letter numeral or generational word anchors as any suffix word does (`John Smith PhD III Ma` reads suffix 'PhD III Ma', `abdul Smith Jr Ma` suffix 'Jr Ma'). ACCEPTED COSTS, each a reading this design chooses. `John Smith, Ms Ma` → suffix 'Ms Ma', reported, as `John Smith, Ms` alone already reads suffix (8). `Om Jr Ma` → given 'Om', suffix 'Jr Ma' and no family name: 'Jr' is not the leading piece, so it anchors 'Ma', and rules/S2's standing Accepted clause already consumes an unambiguous suffix that leaves no family (`Om Jr` → given 'Om', suffix 'Jr'); every release from 2.0.0 made that same role assignment. `doe, jane v phd do` reports `suffix-or-name` where it reported `particle-or-given`, the fork reported being the credential pick rather than P6's attachment (7). And a one-word family comma whose given part OPENS with a capitalised `MA` in a mixed-case name, with an unambiguous credential behind it, reads the whole part as a credential run, family 'Smith' and no given name, where the parent kept given 'MA': `Smith, MA PhD Ma` read given 'MA', middle 'Ma', suffix 'PhD' and reads suffix 'MA PhD Ma'. 'MA' leans credential on its capitals, so it is read as the credential it is written as, and what follows it is then the company's (or, for `M.Eng.`, the chunked gate's). Measured 2026-09-28 on the grid below: 138 parses (46 texts, three orders each) go from given 'MA' to no given name, 72 of them with no `M.Eng.` in the run; 12 of the 138 are the parses the attribution below leaves unattributed, the rest falling in the anchor and `M.Eng.` classes. Each reports 'MA' at the first slot, and a member behind the credential reports as the company's pick (9). ACCEPTED LIMITS, each keeping the member a name word, rules/S2's Accepted block and case rows: the merged `Ph. D.` is outside the no-comma walk (`John Smith Ph. D. MEng` family 'MEng'; the comma and given-slot spellings do anchor it, `Smith, Ph. D. MEng` and `Doe, Jane Ph. D. MEng` reading suffix 'Ph. D. MEng'), a title between the credential and the member breaks the run (`John Smith PhD Prof. Ma`), and a particle member P2 has chained before the peel is out of reach (`John Smith PhD Do Do` family 'Do Do'; `Smith, PhD Do Ma` given 'PhD', middle 'Do Ma'). diff --git a/docs/design/decisions/vocabulary-model.md b/docs/design/decisions/vocabulary-model.md index 8369d462..ba94c2d1 100644 --- a/docs/design/decisions/vocabulary-model.md +++ b/docs/design/decisions/vocabulary-model.md @@ -101,7 +101,7 @@ Open: #752 (phase 2, derived ambiguity), #789 (whether the dotted `Wm.`, `Chas.` - **The title keep-outs became exclusions**, in the file of the language whose reading they refuse, with their counts as name readings where counts were taken (malam, bay, shehu): `uv run python tools/vocab/query.py --excluded title`. The per-word counts in decisions/salutation-titles moved onto the `note` of each word's title reading, and the prose points at the query. The reasons the given-addressing comments gave for leaving `lord`, `lady`, `venerable`, `rabbi` and `imam` out of GIVEN_NAME_TITLES are notes on those words in und.toml until their kinds are assigned (they moved with them in part 2, the same day). - **Recompute** rather than quoting: `uv run python tools/vocab/query.py --kind salutation --count`, `--kind title --file und --count` (the titles still in `und.toml`, all of them generic until part 2 emptied it the same day, leaving one generic title, `wm`, in en.toml; `--kind title` names the whole category, so no query counts the generic kind alone), `--excluded title`. -- **2026-10-19 #750 — the rest of the titles classified (part 2 of #750)** (its `wm` decision superseded by 2026-10-22 #789, and its measurement corrected in place). No compiled set moves: `tools/vocab/build.py --check` reports the committed module current, and `tools/differential/snapshot.py` matches. The 644 titles still generic in `und.toml` on e943665e took their kinds, all but `wm` (below), and left for the files of the languages whose names they occur in, so `und.toml` holds no title. Part 1's hard cases are applied as decided there: `lord`, `lady`, `knight` and `chief` are `nobility`, `deputy` an `office`, and `dr` and `prof` are in mul.toml. +- **2026-10-19 #750 — the rest of the titles classified (part 2 of #750)** (its `wm` decision superseded by 2026-10-22 #789, its `md` academic reading by #853, decisions/indic-honorifics, and its measurement corrected in place). No compiled set moves: `tools/vocab/build.py --check` reports the committed module current, and `tools/differential/snapshot.py` matches. The 644 titles still generic in `und.toml` on e943665e took their kinds, all but `wm` (below), and left for the files of the languages whose names they occur in, so `und.toml` holds no title. Part 1's hard cases are applied as decided there: `lord`, `lady`, `knight` and `chief` are `nobility`, `deputy` an `office`, and `dr` and `prof` are in mul.toml. - **English holds most of them**: the bulk import's occupations, the US military ranks and their abbreviations, the offices, and the modifiers. `modifier` takes what is a title only inside a phrase: the qualifiers of ranks and offices (`vice`, `rear`, `petty`, `field`), the domain words of the descriptive titles (`baseball`, `software`, `tax`), the ordinals, and the court words of #9's list of judges' titles (`designated`, `docket`, `pro`, `edmi`). - **Address, not subject, decides `academic`.** Besides the doctors and professors of every script, it takes the English words used to address a professional (`advocate`, `attorney`, `engineer`, `nurse`, and `md` as the medical degree written before a name), the German abbreviations written before a name (`ra`, Rechtsanwalt; `wp`, Wirtschaftsprüfer; `pd` and `priv.-doz`; `dipl.-ing`), and the Devanagari `पंडित` and `पं`, a scholar's title, as part 1's refusal of the Latin `pandit` already keyed it. - **`wm` keeps the generic kind**, in en.toml: it is William's abbreviation, made a title for #68 so that `Wm. Scott Magargee III` would not read given `Wm.`, and no title kind describes it. Removing it moves a parse, though only of the bare spelling: `Wm Smith` reads title `Wm` with the word listed and given `Wm` without it, while the period-abbreviation shape rule takes `Wm.` either way (measured 2026-10-19 with `Lexicon.default().remove(titles={'wm'})`). So it is left to its own change. (Corrected 2026-10-24, #789: the measurement was wrong on its date. It took the off-switch parse at the head of the name only; a trailing `Wm.`, rules/H5's since 2.3.0, and a `Wm.` before a family comma, rules/H6's since 2.5.0, read vocabulary and move with the bare spelling.) diff --git a/docs/locales.rst b/docs/locales.rst index e23e1868..c4989248 100644 --- a/docs/locales.rst +++ b/docs/locales.rst @@ -364,7 +364,9 @@ by ``tests/v2/test_locales.py``: any. Vocabulary that is *self-selecting* — able to match only text of the tradition it came from, the way a hangul surname can only ever match hangul — is default-safe, and belongs in the default lexicon - (``nameparser/config/``) rather than in a pack: a pack nobody knows + (``nameparser/vocab/*.toml``; a segmentation surname list such as + ``config/surnames.py`` stays in ``nameparser/config/``) rather than in + a pack: a pack nobody knows to ask for is vocabulary nobody gets. Vocabulary that *declares a language its script does not* — a Chinese surname list, which silently mangles the Japanese names written in the same characters diff --git a/docs/migrate.rst b/docs/migrate.rst index 89112d18..e169c77b 100644 --- a/docs/migrate.rst +++ b/docs/migrate.rst @@ -59,6 +59,9 @@ about either: holding whitespace no word carries, in a vocabulary set or as a ``capitalization_exceptions`` key — is dropped with a ``UserWarning`` naming it, since 2.4 +* a ``Lexicon``, ``Policy`` or ``Parser`` pickled by 2.5 or earlier + raises ``ValueError`` on load since 2.6, each having gained a field; + re-pickle after upgrading A silent change: comparing with ``==`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -289,7 +292,9 @@ through the 2.0 API only. Renamed word lists (2.2) ~~~~~~~~~~~~~~~~~~~~~~~~ -The vocabulary that feeds both columns lives in ``nameparser.config``, +The vocabulary that feeds both columns is exposed by ``nameparser.config`` +(compiled since 2.6 from ``nameparser/vocab/*.toml``, all but the Korean +surname list), and in 2.2 its module and constant names moved to the vocabulary the ``Lexicon`` column speaks — particles, bound given names, given-name titles, suffix words. Terminology only; one of the four kept 1.x's @@ -385,13 +390,13 @@ could hold two disagreeing defaults with nothing to say so. ``CAPITALIZATION_EXCEPTIONS`` is the constant the freeze left out. It is a mapping rather than a set, and it is still a plain mutable -``dict`` — ``CAPITALIZATION_EXCEPTIONS["dphil"] = "DPhil"`` runs on 2.2 and +``dict`` — ``CAPITALIZATION_EXCEPTIONS["dclinpsy"] = "DClinPsy"`` runs on 2.2 and raises nothing. Everything just said about split defaults still applies -to it, unchanged and measured on 2.2: an edit after the first parse +to it, unchanged and measured on 2.2 and again on 2.6: an edit after the first parse reaches a freshly built ``Constants``, and neither ``Lexicon.default()`` nor the shared ``CONSTANTS``. The advice below is the same advice — configure the object, with -``constants.capitalization_exceptions["dphil"] = "DPhil"`` on a private +``constants.capitalization_exceptions["dclinpsy"] = "DClinPsy"`` on a private ``Constants``, or for the 2.0 API a ``dataclasses.replace()`` that extends the default pairs rather than replacing them (:ref:`case-exceptions`). @@ -649,12 +654,12 @@ in tokenization rather than by policy, so a name written with either still divides at the dot, and still renders with a space, whatever those two fields are set to. -Changed in 2.3, 2.4 and 2.5 -~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Changed in 2.3, 2.4, 2.5 and 2.6 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Later releases moved more 1.4 readings, and these reach ``HumanName`` as well; 2.1 and 2.2 read every one of them as 1.4 did. The release -log's 2.3, 2.4 and 2.5 sections carry the reasoning; these are the shapes to +log's 2.3, 2.4, 2.5 and 2.6 sections carry the reasoning; these are the shapes to grep your fixtures for, with what 1.4 read: - ``"Her Majesty Queen Elizabeth"``, ``"Prince Harry"`` (2.3) — a lone @@ -722,6 +727,25 @@ grep your fixtures for, with what 1.4 read: are titles: 1.4 read first ``Bapak``, middle ``Dwi``. - ``capitalize()`` on ``"jacobus henricus van 't hoff"`` (2.5) — 1.4 wrote ``van 'T Hoff``. +- ``"Maria de Sa"`` and ``"John Smith SA"`` (2.6) — ``Sa`` is no longer + a credential: 1.4 read last ``de``, suffix ``Sa``, and suffix ``SA``. +- ``"Cllr John Smith"`` and ``"Dott.ssa Anna Bianchi"`` (2.6) — ``Cllr`` + and the Italian abbreviated salutations are titles: 1.4 read first + ``Cllr`` and ``Dott.ssa``, middle ``John`` and ``Anna``. +- ``"José Silva Júnior"`` (2.6) — 1.4 read middle ``Silva``, last + ``Júnior``. +- ``"John Smith, MD; PhD"`` (2.6) — a semicolon between post-nominals + reads as a comma: 1.4 read first ``MD;``, last ``John Smith``. +- ``"Mr Md Karim Uddin"`` and ``"Wm Smith"`` (2.6) — ``Md`` and ``Wm`` + are the given names they abbreviate: 1.4 read title ``Mr Md`` and + ``Wm``. +- ``"Doe, Jane Mrs"`` (2.6) — a salutation ending the given name after a + family comma is the title: 1.4 read middle ``Mrs``. +- A zero-width space or byte-order mark (2.6) divides words as a space + does: 1.4 kept it in the word beside it, so ``"\u200bDr. John Smith"`` + read first ``Dr.``. +- ``capitalize()`` on ``"j.r.r. tolkien"`` (2.6) — 1.4 wrote ``J.r.r.``; + repair now capitalizes after a full stop or an ayn. .. doctest:: @@ -798,3 +822,28 @@ grep your fixtures for, with what 1.4 read: >>> vant.capitalize(force=True) >>> str(vant) "Jacobus Henricus van 't Hoff" + >>> HumanName("Maria de Sa").last + 'de Sa' + >>> HumanName("John Smith SA").last + 'SA' + >>> HumanName("Cllr John Smith").title + 'Cllr' + >>> HumanName("Dott.ssa Anna Bianchi").title + 'Dott.ssa' + >>> HumanName("José Silva Júnior").suffix + 'Júnior' + >>> HumanName("John Smith, MD; PhD").suffix + 'MD; PhD' + >>> md_karim = HumanName("Mr Md Karim Uddin") + >>> md_karim.title, md_karim.first + ('Mr', 'Md') + >>> HumanName("Wm Smith").first + 'Wm' + >>> HumanName("Doe, Jane Mrs").title + 'Mrs' + >>> HumanName("\u200bDr. John Smith").title + 'Dr.' + >>> tolkien = HumanName("j.r.r. tolkien") + >>> tolkien.capitalize() + >>> str(tolkien) + 'J.R.R. Tolkien' diff --git a/docs/release_log.rst b/docs/release_log.rst index b4e4924b..c7125517 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -1,8 +1,13 @@ Release Log =========== -* 2.6.0 - Unreleased +* 2.6.0 - October 25, 2026 - nameparser 2.6 is under development. + nameparser 2.6 ships the default vocabulary as per-word data files, adds an opt-in reading for names with two family names, and fixes words read as a credential or a title that are part of the name. Highlights: + + - **An opt-in reading for names with two family names.** ``Policy(two_family_names=True)`` reads ``Gabriel García Márquez`` with family ``García Márquez``. + - **Surnames that are also credentials read as surnames.** ``Luo Si`` and ``Maria de Sa`` give family ``Si`` and ``de Sa``, where 2.5 read ``Si`` and ``Sa`` as credentials. + - **Wm and Md read as the given names they abbreviate.** ``Wm Smith`` and ``Mr Md Karim Uddin`` give first ``Wm`` and ``Md``, where 2.5 read both as titles. + - **More credentials after a comma read as suffixes.** ``John Smith, MBChB`` gives suffix ``MBChB``, where 2.5 gave first ``MBChB``, last ``John Smith``. **Additions** @@ -17,13 +22,14 @@ Release Log - **Fix case repair writing a joined credential like a hyphenated name.** ``john smith, kchs/dchs`` repairs to ``John Smith KCHS/DCHS``, where 2.0.0 through 2.5.0 gave ``Kchs/Dchs``; a listed acronym joined by a hyphen or slash is written in capitals as one word, and a casing the caller records for it wins. Hyphenated names such as ``Smith-Jones`` keep their per-part casing. :issue:`808` - **Add Cllr as a title.** ``Cllr John Smith`` gives title ``Cllr``, first ``John``, last ``Smith``, where 1.4.0 through 2.5.0 gave first ``Cllr``, middle ``John``. A bare ``Cllr`` after the name is still a name word. :issue:`847` - **Fix case repair lowering a capital after a full stop or an ayn.** ``j.r.r. tolkien`` repairs to ``J.R.R. Tolkien`` and ``AHMAD IBN-ʿALĪ`` to ``Ahmad ibn-ʿAlī``, where 1.4.0 through 2.5.0 gave ``J.r.r.`` and ``ʿalī``. A joined word is written the same way (``st.john`` gives ``St.John``), so an unlisted abbreviation's ending is capitalized too: ``dr.philol.`` gives ``Dr.Philol.``. :issue:`838` - - **Add the Italian abbreviated salutations such as Sig.ra and Dott.ssa as titles.** ``Dott.ssa Anna Bianchi`` gives title ``Dott.ssa``, first ``Anna``, where 1.4.0 through 2.5.0 gave first ``Dott.ssa``, middle ``Anna``, and ``SIG.RA MARIA ROSSI`` repairs to ``Sig.ra Maria Rossi``. A bare ``Sig`` or ``Ing`` before a name is still read as a first name. :issue:`840` + - **Add the Italian abbreviated salutations such as Dott.ssa and Sig.na as titles.** ``Dott.ssa Anna Bianchi`` gives title ``Dott.ssa``, first ``Anna``, where 1.4.0 through 2.5.0 gave first ``Dott.ssa``, middle ``Anna``. A bare ``Sig`` or ``Ing`` before a name is still read as a first name. :issue:`840` - **Add Júnior as a suffix.** ``José Silva Júnior`` gives last ``Silva``, suffix ``Júnior``, as ``José Silva Junior`` does, where 1.4.0 through 2.5.0 gave middle ``Silva``, last ``Júnior``. ``Filho``, ``Neto`` and ``Sobrinho`` still read as the last name. :issue:`843` - **Read a semicolon between post-nominals as a comma.** ``John Smith, MD; PhD`` gives first ``John``, last ``Smith``, suffix ``MD; PhD``, where 1.4.0 through 2.5.0 gave first ``MD;``, last ``John Smith``. A delimiter set in ``extra_suffix_delimiters`` counts the same way when glued to a suffix word or written between two (``MD/PhD``), and a semicolon before the first comma (``John Smith MD; PhD``) is still read as part of a word. :issue:`871` - **Read a leading Md as the given name it abbreviates, Muhammad, rather than a title.** ``Mr Md Karim Uddin`` gives title ``Mr``, first ``Md``, middle ``Karim``, where 1.4.0 through 2.5.0 gave title ``Mr Md``, first ``Karim``; ``John Smith, MD Jones`` gives last ``Smith``, suffix ``MD Jones`` where it gave title ``MD``, last ``John Smith``. ``MD`` after a name is still the degree, and ``Md.`` with a full stop is still a title. One limit: ``Md`` opening the part after a family comma reads as a credential, so ``Uddin, Md Karim`` gives suffix ``Md Karim`` where it gave title ``Md``, first ``Karim``. :issue:`853` - **Read Wm as the given name it abbreviates, William, rather than a title.** ``Wm Smith`` gives first ``Wm``, last ``Smith``, where 1.4.0 through 2.5.0 gave title ``Wm`` with no first name. ``Wm.`` with a full stop opening the name is still a title. :issue:`789` - **Fix a surname written as two dotted initials reading as a credential.** ``Ram Bahadur B.K.`` gives middle ``Bahadur``, last ``B.K.``, where 2.4.0 through 2.5.0 gave last ``Bahadur``, suffix ``B.K.``, restoring 2.3.0's reading, and ``Hari G.C.`` gives last ``G.C.``, where 2.0.0 through 2.5.0 gave suffix ``G.C.`` with no last name. Three or more dotted letters after a full name are still a credential (``John Smith X.Y.Z.``), and so is a listed one (``John Smith M.A.``). :issue:`864` - - **Read a salutation ending the first name after a family comma as the title.** ``Doe, Jane Mrs`` gives title ``Mrs``, first ``Jane``, last ``Doe``, where 1.4.0 through 2.5.0 gave middle ``Mrs``; ``Doe, Jane Mrs.`` has read as the title since 2.3.0. A salutation also borne as a name, such as ``Goodman`` or ``Herr``, is still read as a middle name, and so is any other title, such as ``King`` or ``Dr``; ``Lexicon.after_given_titles`` lists the salutations. A ``Lexicon`` or ``Parser`` pickled by an earlier release no longer loads; re-pickle after upgrading. :issue:`835` + - **Read a salutation ending the first name after a family comma as the title.** ``Doe, Jane Mrs`` gives title ``Mrs``, first ``Jane``, last ``Doe``, where 1.4.0 through 2.5.0 gave middle ``Mrs``; ``Doe, Jane Mrs.`` has read as the title since 2.3.0. A salutation also borne as a name, such as ``Goodman`` or ``Herr``, is still read as a middle name, and so is any other title, such as ``King`` or ``Dr``; ``Lexicon.after_given_titles`` lists the salutations. :issue:`835` + - **A Lexicon, Policy or Parser pickled by an earlier release no longer loads.** Loading one raises ``ValueError`` naming the new field (``after_given_titles`` or ``two_family_names``); re-pickle after upgrading. ``ParsedName`` and ``HumanName`` pickles still load. :issue:`754`, :issue:`835` - **Fix a zero-width space or byte-order mark sticking to the word beside it.** ``Dr. John Smith`` with an invisible zero-width space in front gives title ``Dr.``, first ``John``, where 1.4.0 through 2.5.0 gave first ``Dr.`` (the character kept), middle ``John``. The two now divide the words of a name as a space does, with no setting to turn that off, though not inside quotes or a ``(née ...)`` clause; the zero-width joiner and non-joiner still stay inside the word. :issue:`851` * 2.5.0 - October 19, 2026 diff --git a/docs/usage.rst b/docs/usage.rst index 3479a8ee..24a6638f 100644 --- a/docs/usage.rst +++ b/docs/usage.rst @@ -48,7 +48,10 @@ name in forms 1 and 3 and at the end of the given name in form 2; written bare there (``Prof``), it is a name word, because many title words are also surnames — unless a credential stands before it, which starts a run of post-nominals and titles (``John Smith PhD Prof`` reads -title ``Prof``; see :ref:`trailing-titles`). +title ``Prof``; see :ref:`trailing-titles`), or, at the end of the +given name in form 2, it is a listed salutation such as ``Mrs`` +(``Doe, Jane Mrs`` reads title ``Mrs`` since 2.6; +``Lexicon.after_given_titles`` lists them). The last two differ in what the comma is doing. In form 2 it separates the family name from the rest, so the family name comes first; in form diff --git a/nameparser/_pipeline/_assign.py b/nameparser/_pipeline/_assign.py index 1fdbb38d..b6305eca 100644 --- a/nameparser/_pipeline/_assign.py +++ b/nameparser/_pipeline/_assign.py @@ -15,7 +15,7 @@ across pieces); token/piece tags; Lexicon only through tags already applied by classify (plus the leading-title period rule). -Implements rules H2, H4, H5, H6, H7, N3, O4, O5, S10, W4, and P6 after a family comma, +Implements rules H2, H4, H5, H6, H7, H8, N3, O4, O5, S10, W4, and P6 after a family comma, of docs/design/rules/, each cited at its code below. Ports v1's assignment loops. NO_COMMA (per name_order): diff --git a/nameparser/_pipeline/_comma.py b/nameparser/_pipeline/_comma.py index 498df9a6..7d5147dc 100644 --- a/nameparser/_pipeline/_comma.py +++ b/nameparser/_pipeline/_comma.py @@ -22,8 +22,8 @@ _vocab.caps_shape_candidate (to know an all-caps word is unlisted) and _vocab.claimed_as_non_name (the name's contrast); Lexicon .maiden_markers through _pieces.own_words, for the name's contrast; -Policy.lenient_comma_suffixes, Policy.extra_suffix_delimiters and -Policy.unlisted_caps_suffixes. +Policy.lenient_comma_suffixes, Policy.extra_suffix_delimiters, +Policy.unlisted_caps_suffixes and Policy.unlisted_dotted_suffixes. """ from __future__ import annotations diff --git a/nameparser/_pipeline/_post_rules.py b/nameparser/_pipeline/_post_rules.py index 4b244a23..1109c4bd 100644 --- a/nameparser/_pipeline/_post_rules.py +++ b/nameparser/_pipeline/_post_rules.py @@ -12,7 +12,8 @@ family comma that names a family, that attachment is assign's since #613. Reads: Policy.patronymic_rules, Policy.two_family_names, -Policy.middle_as_family, +Policy.name_order (O6 stands down where the script's order differs +from the declared one), Policy.middle_as_family, Policy.extra_suffix_delimiters (R1's entry pass, for the delimiter cores group drops); Lexicon.given_name_titles. diff --git a/nameparser/_version.py b/nameparser/_version.py index 74e131ec..cea4abcd 100644 --- a/nameparser/_version.py +++ b/nameparser/_version.py @@ -15,5 +15,5 @@ #: "dev" through the cycle, cleared at release. It normalizes to #: 2.6.0.dev0, which sorts above 2.5.0 and BELOW 2.6.0, so an install #: from master can never masquerade as the release it precedes. -PRE_RELEASE = "dev" +PRE_RELEASE = "" __version__ = ".".join(map(str, VERSION)) + PRE_RELEASE diff --git a/nameparser/config/__init__.py b/nameparser/config/__init__.py index 6cf7183f..e821af01 100644 --- a/nameparser/config/__init__.py +++ b/nameparser/config/__init__.py @@ -10,6 +10,8 @@ ...) are not compatibility surface at all: they are the word lists the 2.0 :class:`~nameparser.Lexicon` is built from, they are named for its fields since 2.2 (#293), and its documentation cross-references them. +Since 2.6 the word lists (all but ``surnames.py``) re-export sets +compiled from the per-word TOML files in ``nameparser/vocab/`` (#737). ``RegexTupleManager`` is re-exported unchanged from the shim purely for pickle compatibility: a v1.4 ``Constants`` blob's ``regexes`` field was diff --git a/nameparser/config/capitalization.py b/nameparser/config/capitalization.py index 2698d8ac..8a8bc74b 100644 --- a/nameparser/config/capitalization.py +++ b/nameparser/config/capitalization.py @@ -30,11 +30,11 @@ suffix the writer spelled in more than one case (``EdD``) needs no entry either: repair keeps it as written unless forced. A caller's OWN acronym -- one absent from ``suffix_acronyms`` -- depends on how -it is written. Written PLAINLY (``dphil``) it parses as an ordinary -name word and repairs as one (``Dphil``). Written DOTTED -(``d.phil.``) it is a suffix by shape alone, with no vocabulary +it is written. Written PLAINLY (``dclinpsy``) it parses as an ordinary +name word and repairs as one (``Dclinpsy``). Written DOTTED +(``d.clinpsy.``) it is a suffix by shape alone, with no vocabulary entry needed to read it as one, and repairs in all capitals the same -as a listed acronym does (``D.PHIL.``). Either way, a caller who +as a listed acronym does (``D.CLINPSY.``). Either way, a caller who wants a specific spelling needs a ``suffix_acronyms`` entry (``Lexicon.add(suffix_acronyms={...})`` in the 2.0 API, ``constants.suffix_acronyms.add(...)`` in the v1 one) or a mask of diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 9ac293ba..2e53f6ce 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -1328,7 +1328,7 @@ def _check_cjk_shape_purity(self) -> None: "John Smith, MD MEng", {"given": "John", "family": "Smith", "suffix": "MD MEng"}, ambiguities=("suffix-or-name",), - notes="the same dual after a full name, before the other " + notes="the same credential after a full name, before the other " "chunked member. 1.4.0 and 2.3.0 read the same suffix; " "#540 had read title 'MD', given 'MEng'", shape=3), @@ -10821,10 +10821,10 @@ def _check_cjk_shape_purity(self) -> None: notes="segment 0 is the family segment even when it is wholly " "credential-shaped, so 'MD' is the FAMILY and not the " "head of a credential run, and that is what this row " - "still guards. Since #701 it guards rules/H6's stop " - "too: 'MD' is title vocabulary, and a title before the " - "comma peels only in front of a family word, which 'PhD' " - "is not. The SEPARATOR is no longer the row's " + "still guards. From #701 until #853 took 'md' out of " + "the titles it guarded rules/H6's stop too, a title " + "before the comma peeling only in front of a family " + "word. The SEPARATOR is no longer the row's " "subject. Until #436 it was: the join was asked of " "segment 1 alone, and dropping that conjunct left the " "whole suite green while this shape's suffix silently " @@ -10856,8 +10856,8 @@ def _check_cjk_shape_purity(self) -> None: {"family": "Smith", "suffix": "MD I"}, classification="fix(#430)", notes="the same shape reached through the leading-title peel " - "instead: md is TITLES vocabulary too (the #296 " - "deviation), so this read title 'MD' + given 'I' where " + "instead: md was TITLES vocabulary too until #853 (the " + "#296 deviation), so this read title 'MD' + given 'I' where " "'Smith, PSM I' read given 'PSM' + suffix 'I'. Two " "wrong answers, one cause -- a fix verified on PSM " "alone would leave this one broken and look green. " diff --git a/uv.lock b/uv.lock index f90ce846..aa4df9ec 100644 --- a/uv.lock +++ b/uv.lock @@ -110,11 +110,11 @@ wheels = [ [[package]] name = "certifi" -version = "2026.5.20" +version = "2026.10.23" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/f3/ce/ee2ecad540810a79593028e88299baeae54d346cc7a0d94b6199988b89b1/certifi-2026.5.20.tar.gz", hash = "sha256:69dea482ab64caa7b9f6aba1c6bf48bb6a5448d1c0f1b17ab42ad8c763a5344d", size = 135422, upload-time = "2026-05-20T11:46:50.073Z" } +sdist = { url = "https://files.pythonhosted.org/packages/fd/11/b524cbd9bbdaca58608e4cf4053fd463417549e1959fa8d3aa18319f9555/certifi-2026.10.23.tar.gz", hash = "sha256:a9895e6e2758b371e0545e321a3a8fd1fcfd9829ada8824a936dfb16aaaf36cd", size = 136225, upload-time = "2026-10-23T11:36:16.478Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/59/8c/57e832b7af6d7c5abe66eb3fbe3a3a32f4d11ea23a1aa7131371035be991/certifi-2026.5.20-py3-none-any.whl", hash = "sha256:3c52e209ba0a4ad7aebe60436a4ab349c39e1e602e8c134221e546902ad25897", size = 134134, upload-time = "2026-05-20T11:46:48.578Z" }, + { url = "https://files.pythonhosted.org/packages/32/d1/a3652fe2c19134d0d15922177821d863cbd7e842d53a21bbe7e8e3a1b5ac/certifi-2026.10.23-py3-none-any.whl", hash = "sha256:e9b54de76e530bb6cf0ccd036787103acb58fd42ba930ea8bf948ddbaa2f61d1", size = 135082, upload-time = "2026-10-23T11:36:14.763Z" }, ] [[package]]