reader

A pure-Go, zero-C PDF reader — the parsing half of go-pdfkit. Where pdfkit authors new PDF 1.7, this module takes existing PDF bytes apart: lexer, object model, cross-reference tables and streams, stream filters, the standard security handler, the page tree, and content streams. It also writes the same object graph back out, which is what every operation on an existing file needs. Nothing outside the Go standard library is required, so it builds for GOOS=js/wasm and every 64-bit architecture the fleet targets.

Status

Landing in waves. This module currently provides the object layer and the document structure:

• Lexer — the full PDF object syntax: integers and reals in the shapes producers actually write, literal strings with their escapes, hex strings, names with #xx escapes, arrays, dictionaries, streams, and keywords.
• Objects — Null, Bool, Integer, Real, String, Name, Array, Dict, Stream, and indirect Ref, with the N G R / N G obj lookahead resolved the way the specification requires.
• Filters — FlateDecode, LZWDecode (with PDF's EarlyChange, which the standard library's LZW does not implement), ASCIIHexDecode, ASCII85Decode, RunLengthDecode, Crypt, plus the PNG and TIFF predictors. Image filters (DCTDecode, JPXDecode, CCITTFaxDecode, JBIG2Decode) are reported rather than applied, so an image consumer gets the encoded bytes.
• Filters that fail — a chain that cannot be run to the end is not the end of the stream. DecodeRecovering returns what the filters before the failure produced — the prefix a damaged Flate stream did inflate, not its compressed bytes — and says so through Decoded.Recovered and Decoded.Cause. Bytes no filter decoded arrive in Decoded.Undecoded, never in Decoded.Data, so a compressed stream cannot be painted as samples or tokenised as content by a caller that forgot to check a flag. Decode is the strict reading and refuses such a stream outright, so a caller that must not act on damaged data says which it wants by which one it calls. Page content takes the lenient reading, because a page whose content will not decode is still a page; PageContentDecoded reports whether any salvaging happened. Cross-reference streams take the strict one: a table that has to be guessed at is worse than no table, and the repair below is the answer to one.
• Cross-references — classic tables, cross-reference streams, object streams, /Prev chains, and the /XRefStm of a hybrid file, newest definition winning.
• Repair — a file whose tables are missing, truncated or simply wrong is rebuilt by scanning it for object headers, trailers and, failing those, for a catalogue; a file that kept its pages but lost its catalogue gets one. This is not an exceptional path: it is what makes a reader usable. The rebuild establishes the file key before it reads object streams, so a file whose catalogue lives in an encrypted one can be rebuilt at all; it takes /Encrypt from a cross-reference stream's dictionary when the file has no trailer keyword to name it in. Where two object streams claim the same object, the later one in the file wins, the same rule the header scan applies to objects written directly — so the same damaged file reads the same way twice. When a rebuild fails, the error says what the rebuild found and not merely that the tables could not be read.
• Documents — Open, object resolution with cycle and recursion guards, the trailer, the catalogue, and the page tree with the four attributes a page inherits from its ancestors.
• Encryption — the standard security handler in every revision, both ways: reading RC4 at 40 and 128 bits, AESV2, and the AES-256 of /R 5 and /R 6, with crypt filters, /Identity and /EncryptMetadata; and writing AES-256 or AES-128, with permissions, a user password and an owner password. A password is tried as the user password and as the owner password, and Open uses the empty one, so a file protected only against editing opens with no password at all. and steps over rubbish rather than losing the operations around it, with inline images read whole. Where an inline image ends is the one genuinely ambiguous thing in a content stream, since its data may spell EI itself; the length is computed from the image's own geometry where it can be, and otherwise every candidate EI is tried until the data before one actually decodes.
• Writing — the exact inverse of the parser: objects rendered in PDF syntax with stable dictionary ordering and strings in whichever of the two forms is shorter, and a Writer that copies whole object graphs out of one or more documents, renumbering as it goes, then lays down a cross-reference table and a trailer. NewPackedWriter instead puts what it can into compressed object streams and ends in a cross-reference stream, which is what makes a file small. Measured against a corpus of 118 863 real PDFs — Matplotlib, cairo, pdfTeX, Ghostscript, Adobe, R, Apache FOP, PDF 1.3 through 1.7 — Open succeeds on 118 833 of them and finds 138 337 pages, in 8 seconds, with no panics. Of the thirty it refuses, twenty-seven are PNG files with a .pdf extension, one is PostScript, and two are PDFs truncated past recovery.

The one encrypted PDF the corpus holds — /V 4 /R 4, written by macOS Quartz — decrypts to readable metadata and a valid content stream, which is the only genuinely independent check of the key derivation there is here; the other revisions are round-tripped against a test encryptor written from the same algorithms.

The content-stream tokeniser reads 1 536 769 753 operations across those 138 337 pages in a minute and a half, with no panics, no page failing to decode, and no operator outside the seventy the format defines — beyond four arc and six nan written by a producer that was simply wrong.

Rewriting is checked the same way: every one of the 118 833 files that open is copied object by object into a new file, which is then re-read and compared on page count, media boxes and the bytes of every content stream. All 118 833 match, 35.5 GB in and 35.2 GB out, in twenty-eight seconds — and all 118 833 match again when the packed writer is used instead, which brings the output down to 94.6% of the input.

Linearization (Fast Web View)

Linearized(b) reads the linearization parameter dictionary at the front of a file (ISO 32000-1 Annex F, Table F.1) and checks it against the file. A linearized PDF is ordinary PDF rearranged so a reader with a byte-range transport can show the first page before the rest has arrived.

⛔ It is not a yes/no. Table F.1 says a file-length mismatch "indicates that the file is not linearized and shall be treated as ordinary PDF", and F.3.6 says any incremental update invalidates the hint tables — which is the ordinary fate of these files. So the three errors mean different things: ErrNotLinearized (never was), ErrLinearizationMalformed (breaks a rule of Table F.1), ErrLinearizationStale (was, and the file changed underneath it).

Checked: the dictionary is the first object of the body — not "one found somewhere near the front", because searching would turn up a dictionary left by an earlier write and call the file linearized when what a reader looks at first is something else; that it lies entirely within the first 1024 bytes, the rule that makes the format worth having; that every value is direct, so no cross-reference table is needed to read it; the required entries and their types; and /L against the real file length.

Not checked: the hint tables themselves (F.4) — a file can pass this and still send a reader to the wrong offset.

The positives are qpdf's output, not fixtures written to agree with this reading of Annex F. On five real library scans linearized by qpdf and judged by qpdf --check alongside: 5 agree, 0 disagree, including a 68-page, 13 MB document.

⛔ Reading is all it does — there is no linearizing writer yet. The checker came first on purpose: a writer cannot be trusted without one.

Next wave: the operations built on all of this.

Install

go get github.com/go-pdfkit/reader

Testing

go test -covermode=set ./...

CI gates on exact 100% statement coverage, go vet, and a cross-compile across linux/{amd64,arm64,riscv64,loong64,ppc64le,s390x}, js/wasm, darwin/arm64 and windows/amd64.

The rest of the family

reader reads and writes the format · ops is the verbs and the pdfops command · render turns a page into pixels · pdffont is what a document says about a font · extract reads a page back · coedit is a PDF several people edit at once · app is the browser workbench.

Documentation for all of it: https://go-pdfkit.github.io/docs/

License

BSD-3-Clause — see LICENSE. Copyright the go-pdfkit/reader authors.