[tannevaled]

A PDF page has no headings, no paragraphs and no lists. It has instructions for
drawing glyphs at places, and everything a reader sees above that is an
inference. This turns those inferences into the
richdoc model — which gets Markdown,
reST and HTML out of a PDF through code the fleet already has.

main was created by GitHub (ghnew), so no unreviewed commit of ours is in
the history.

What a heading is

Text set larger than the body, where the body size is the one the document
draws the most characters at, over every page. That much is PyMuPDF4LLM's
IdentifyHeaders rule.

⛔ But larger is not enough, and taking it as enough was a defect this
reported on four real documents out of six. A 9-page report set in 10, 12 and
14 point came back with 131 headings, 130 of which were paragraphs — the
report has two body sizes, and "most popular" can only pick one.

So a heading size also has to be rare. Measured over six real documents, as
the share of characters at each size above that document's body size:

heading sizes	0.01% – 5.76%
second body sizes	32.6% – 39.4%

An order of magnitude apart; the threshold sits between them at 15%, and the
code says why that is a fact about documents rather than a tuning knob.

⛔ The order of the whole thing

A numbered heading leaves a gap after its number; the table below it leaves
that same gap empty on every row; so asking the page for its tables returns
one two-column table that is the heading and the grid together — and the
grid's real columns are gone. Page 2 of a 6-page paper read as:

| 2. | Vue d’ensemble technique |
| --- | --- |
|  | HGX B300 GH200 NVL32 GB200 NVL36 GB300 NVL72 |

Nothing in the alignment says that line is a heading; its size does, and
this package is the only one that knows which sizes those are. So the tables
are looked for among the lines that are not headings, and the same page now
reads:

### 2. Vue d’ensemble technique

| HGX B300 | GH200 NVL32 | GB200 NVL36 | GB300 NVL72 |
| --- | --- | --- | --- |

No error

Nothing here can fail. The document has already been parsed to get this far,
and a page that cannot be read is an empty page — an answer, not a failure.
An error that is always nil teaches every caller to write a branch that never
runs.

How it is checked

100% of statements with no uncovered blocks, 21 mutations caught. The one
survivor is the tie-break between two equally popular sizes, which buys
determinism and not a different reading; the code says so where it is.

Several tests needed fixtures with real proportions — a document of one
heading and one short line makes that heading a fifth of the text, and the
share rule correctly calls a fifth of a document body text. Two tests failed on
that, and the fixtures were what was wrong. One test passed against the
unfixed code because its heading sat six points from its number, too narrow
to be a boundary at all; it now asserts the defect before asserting the fix.

Measured on real documents, the control last:

                           pages     h    par    ul/li     tbl
a 213-page reference work    213    695   1257  334/1989    30
a 9-page report                9      1     43    6/21       4
a 6-page technical paper       6     11     29    3/10      12
a 2-page note                  2      7     12    1/3        2
a 3-page prose report          3      0     12    3/6        0

Parity

This is the capability behind BentoPDF's pdf-to-markdown and
prepare-pdf-for-ai; the pdfops verbs come next.

🤖 Generated with Claude Code