diff --git a/CHANGELOG.md b/CHANGELOG.md index fa14d44..c4e4644 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -39,7 +39,16 @@ against an empty feed. keeps the stored JPEG and `DecodedPixels` carries the samples, so a caller can have either. Baseline and extended sequential (SOF0/SOF1) are decoded, and a subsampled component is upsampled with libjpeg's 3:1 fancy filter, so a 4:2:0 fixture matches a reference decoder within two levels. Progressive (SOF2) - is the next step, per ADR-005. + followed in the next entry. +- **Progressive JPEG images are decoded.** `DCTDecode` now reads the multi-scan form too: spectral + selection and successive approximation accumulate in the quantised coefficient grids the scans share, + and the inverse DCT runs once at the end, so a non-interleaved AC scan (one component) and an + interleaved DC scan (every component, MCU order) land on the same blocks. Seven progressive fixtures — + 4:4:4, 4:2:0, grayscale, CMYK, a non-square 32 x 16 image, a sparse one that uses the EOB and zero runs, + and one with 150 restart markers — each matches Pillow within two levels of every sample. The AC + refinement's two conventions — the sign bit is read before the correction bits, and 1 means positive — + are recorded in [`ADR-005`](docs/adr/0005-image-filter-decoders.md), because getting either backwards + still decodes every scan and leaves a handful of coefficients a step or two out. ## [1.8.0] - 2026-11-14 diff --git a/docs/adr/0005-image-filter-decoders.md b/docs/adr/0005-image-filter-decoders.md index 8ba0209..4894d73 100644 --- a/docs/adr/0005-image-filter-decoders.md +++ b/docs/adr/0005-image-filter-decoders.md @@ -1,8 +1,8 @@ # ADR-005: Decoding the image filters the reader passes through - **Status**: **Accepted** (2026-11-14) — `RunLengthDecode` (2026-11-14), `CCITTFaxDecode` (2026-11-15) and - baseline `DCTDecode` (2026-11-15) implemented; progressive JPEG, `JPXDecode` and `JBIG2Decode` next, one - release slice each + `DCTDecode` (baseline 2026-11-15, progressive 2026-11-15) implemented; `JPXDecode` and `JBIG2Decode` + next, one release slice each - **Date**: 2026-11-14 - **Related**: [`docs/parity.md`](../parity.md) row 14, [`docs/manual/src/08-inspection.md`](../../manual/src/08-inspection.md), @@ -42,7 +42,7 @@ Shipping order and acceptance criteria: | --- | --- | --- | | `RunLengthDecode` | — | **Done.** Literal and repeat runs, an optional end-of-data byte, a truncated stream decoded as far as it goes. `PdfImageFilterTests` and the `FilterChains` fixture pin it, and the image is exposed as `Gray8`. | | `CCITTFaxDecode` | Group 3 one-dimensional (`/K` = 0), Group 3 two-dimensional (`/K` > 0), Group 4 (`/K` < 0); `/Columns`, `/Rows`, `/BlackIs1`, `/EncodedByteAlign`, `/EndOfLine` | The decoded bilevel bits equal an independent decoder's output on the same bytes. Evidence: Pillow's CCITT reader through `pikepdf` on the fixture, and a real fax sample in `tests/.../Assets/parity/images`. | -| `DCTDecode` | SOF0, SOF1, SOF2; restart markers; 1, 3 and 4 components; 4:4:4, 4:2:2, 4:2:0; YCbCr and CMYK/YCCK (Adobe APP14); EXIF orientation | Geometry and component count exact; every sample within **2 levels** of a reference decoder (an integer IDCT is allowed to differ by rounding). Evidence: Pillow on the `parity/images/jpeg-*.jpg` fixtures, compared pixel by pixel, with the per-image maximum recorded in `docs/mutation-testing.md`'s sibling measurement note. | +| `DCTDecode` | SOF0, SOF1, SOF2; restart markers; 1, 3 and 4 components; 4:4:4, 4:2:2, 4:2:0; YCbCr and CMYK/YCCK (Adobe APP14); EXIF orientation | Geometry and component count exact; every sample within **2 levels** of a reference decoder (an integer IDCT is allowed to differ by rounding). Evidence: Pillow on the `tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-*.jpg` fixtures — baseline 4:4:4, grayscale, 4:2:0, CMYK and restart, plus progressive 4:4:4, 4:2:0, grayscale, CMYK, non-square, sparse and restart-interval — compared pixel by pixel, with the per-image maximum recorded in `docs/mutation-testing.md`. | | `JPXDecode` | JP2 and raw J2K codestream; 5/3 reversible and 9/7 irreversible; tiles and tile-parts | Reversible path **bit-exact**; irreversible path within 2 levels. Evidence: `opj_decompress` on the same codestream. | | `JBIG2Decode` | Generic region, symbol dictionary, text region, MMR | Bit-exact bilevel output. Evidence: `jbig2dec` on the same stream. | @@ -127,14 +127,30 @@ document holds — and fills `DecodedPixels` from the new decoder, so a caller t and a caller that wants pixels gets them. `ImagePixels` stops excluding `DCTDecode` once `TryDecodeJpeg` is wired; the budget rules then apply to the pixel buffer as they do to every other decoded filter. -Order: SOF0 (baseline) first, against these fixtures; SOF1/SOF2 (extended and progressive) after it, with +Order: SOF0 (baseline) first, against these fixtures; SOF1 (extended) and SOF2 (progressive) after it, with their own fixtures and the same ±2 tolerance. The baseline decoder is implemented: `PdfJpegFilterTests` decodes both fixtures within two levels of Pillow, and the contract above holds — `Data` is the stored JPEG, `DecodedPixels` the samples. The variants the mutation survivors asked for are there too: `jpeg-420.jpg` (4:2:0, decoded with libjpeg's 3:1 fancy filter and within two levels of Pillow), `jpeg-cmyk.jpg` (four components, inverted as the `/Decode` array -states) and `jpeg-restart.jpg` (restart markers). SOF2 is the next slice. +states) and `jpeg-restart.jpg` (restart markers). + +The progressive decoder followed on 2026-11-15: the scans' quantised coefficients are accumulated in one +grid per component and the inverse DCT runs once at the end, so a non-interleaved AC scan (one component, +its blocks) and an interleaved DC scan (every component, MCU order) both land on the same blocks. Seven +fixtures pin it — `jpeg-progressive.jpg` (32 x 32, quality 95, 4:4:4), `jpeg-progressive-420.jpg` (4:2:0), +`jpeg-progressive-gray.jpg` (one component), `jpeg-progressive-cmyk.jpg` (four components), and +`jpeg-progressive-wide.jpg` (32 x 16, so the block counts differ), `jpeg-progressive-sparse.jpg` (flat +areas, so the EOB runs and zero runs are used) and `jpeg-progressive-restart.jpg` (150 restart markers, +so the predictors and the EOB run reset) — each compared with Pillow's decode within two levels. The +truncated-stream theory cuts the entropy data at nine points and pins the refusals. +Two details cost a debugging round and are written down because a first implementation gets both wrong +without failing: in an AC refinement scan the sign bit of a new coefficient is read **before** the +correction bits of the coefficients the run passes over, and a sign bit of **1 means positive** (0 means +negative) — the inverse of the magnitude-field convention an AC first scan uses. Getting either backwards +decodes every scan to the end and leaves a handful of coefficients one or two steps out, which is why the +acceptance is a pixel comparison and not "it decoded". ## Status of the slices @@ -142,6 +158,6 @@ states) and `jpeg-restart.jpg` (restart markers). SOF2 is the next slice. | --- | --- | | `RunLengthDecode` | **Implemented** (2026-11-14) | | `CCITTFaxDecode` | **Implemented** (2026-11-15) | -| `DCTDecode` | **Baseline (SOF0/SOF1) implemented, fancy upsampling** (2026-11-15); progressive (SOF2) next | +| `DCTDecode` | **Baseline (SOF0/SOF1) and progressive (SOF2) implemented, fancy upsampling** (2026-11-15); `JPXDecode` next | | `JPXDecode` | Planned | | `JBIG2Decode` | Planned | diff --git a/docs/manual/src/08-inspection.md b/docs/manual/src/08-inspection.md index 4e83f0c..041ba53 100644 --- a/docs/manual/src/08-inspection.md +++ b/docs/manual/src/08-inspection.md @@ -233,8 +233,8 @@ component counts, or an `Indexed` space over any of them. An image that carries case rather than an exception — its payload is its samples — so an inline image, which producers rarely compress, states its pixels the way a Flate one does. Nothing is left behind a filter either, because the filters an image realistically uses are decoded here — Flate and LZW, each with its predictor, the two -ASCII encodings, run-length, the CCITT fax codings (Group 3 and Group 4), and baseline JPEG — so an image -that a viewer can show is an image whose samples this record states. +ASCII encodings, run-length, the CCITT fax codings (Group 3 and Group 4), and JPEG — baseline and +progressive — so an image that a viewer can show is an image whose samples this record states. Two things are unpacked on the way out rather than handed over as they lie in the file. Samples narrower than a byte are widened to the same scale, so a bilevel image yields `0` and `255` rather than `0` and `1`, a @@ -250,7 +250,7 @@ which one it is: | Case | What the entry says | Why no pixels | | --- | --- | --- | -| A progressive `DCTDecode` image (SOF2), or a baseline one whose buffer is above the budget | `IsDecodable` true, `Filter` `DCTDecode`, `PixelFormat` null | A baseline JPEG is decoded to `Gray8`, `Rgb24` or `Cmyk32`; progressive is the next variant [`ADR-005`](../../adr/0005-image-filter-decoders.md) schedules, and a buffer above the budget is measured rather than held. | +| A `DCTDecode` image whose buffer is above the budget | `IsDecodable` true, `Filter` `DCTDecode`, `PixelFormat` null | A baseline or progressive JPEG is decoded to `Gray8`, `Rgb24` or `Cmyk32`; a buffer above the budget is measured rather than held. | | A filter the library passes through — `JPXDecode`, `JBIG2Decode`, or a name it does not know | `IsDecodable` false, `Filter` names it, and the read warns with `ImageFilterUnsupported` | Nothing was decoded, so there are no samples to hand back. [`ADR-005`](../../adr/0005-image-filter-decoders.md) schedules these decoders, in that order. | | A `Separation`, `DeviceN` or `Lab` space, sixteen bits per sample, a palette whose samples run past its table or whose table holds nothing, a width or height of zero, or a geometry the document does not state | `BitsPerComponent` or `ColorSpace` states which | The first needs a function evaluated, the second a rounding rule, the third a decision the file does not support, and the last a rectangle that is not there. Pixels are eight bit and channel-ordered by contract, so converting or guessing would hide the mismatch from the caller. | diff --git a/docs/mutation-testing.md b/docs/mutation-testing.md index fcb5107..5309b02 100644 --- a/docs/mutation-testing.md +++ b/docs/mutation-testing.md @@ -899,6 +899,64 @@ The survivors are the **variants the two fixtures do not carry**, not statements in part. Those tests are now in — `jpeg-420.jpg`, `jpeg-cmyk.jpg`, `jpeg-restart.jpg` and a truncated stream — and -the score above predates them. The fancy upsampling that matches Pillow at every edge, and the progressive -variant, are the ADR's next slice. +the score above predates them. The fancy upsampling that matches Pillow at every edge followed, and the +progressive variant's reading is below. + +## DCTDecode — progressive (SOF2) (2026-11-15) + +Scoped run over `Pdf/JpegDecoder.cs` after the progressive scans landed. The command is the 5.0.0 +project-mode form with a copy of `stryker-config.json` that restores the `test-projects` entry, because +5.1.0 in this environment ignores the scope (the note at the end of this section has the evidence): + +```powershell +"%TEMP%\stryker-tools\dotnet-stryker.exe" --config-file stryker-jpeg-config.json ` + --project src\SharpCorePDF.Infrastructure\SharpCorePDF.Infrastructure.csproj ` + --mutate "**/JpegDecoder.cs" --output artifacts\mutation\jpeg-progressive +``` + +20,004 mutants are created for the solution and the filter leaves **839** for the file (710 killed, 127 +survived, 2 timeouts). The report is per file, so the whole decoder is read, not only the new region: + +| File | Score | Killed | Survived | No coverage | Mutants | +| --- | --- | --- | --- | --- | --- | +| `Pdf/JpegDecoder.cs` (baseline, 2026-11-15) | 61.65% | 351 | 114 | 105 | 671 | +| `Pdf/JpegDecoder.cs` (progressive, before the coverage fixtures) | 70.23% | 650 (+1 timeout) | 146 | 130 | 1122 | +| `Pdf/JpegDecoder.cs` (progressive, after them) | **76.81%** | 710 (+2 timeouts) | 127 | 88 | 1122 | + +The coverage fixtures the first reading asked for — the restart interval, the sparse image, the CMYK file +and the non-square grid — and the truncated-stream theory moved the new region (line 520 onwards) from +**72.0% to 82.5%**: 471 killed, 46 survived, 54 without coverage, 71 compile errors and 72 ignored. What +remains is the boundary and error end of the decoder rather than a scan that is not decoded: + +- **The DC size and the end-of-band boundaries** (L689, L700, L953, L960, L975, L998): a valid stream + whose DC difference needs eleven bits, or whose last coefficient sits exactly on `Se`, is the missing + variant. The fixtures' DC differences are small and their bands end before the last coefficient. +- **The DC refinement at Al = 0** (L719): `|=` and `^=` agree while the bit is clear and `1 >> 0` is `1`, + so the fixture cannot tell them apart. A stream that refines the DC at `Ah = 2, Al = 1` does. +- **The YCCK transform** (L1093–L1115): `jpeg-progressive-cmyk.jpg` is a plain CMYK file (Adobe transform + 0); the YCCK branch (transform 2) has no fixture. The baseline record's CMYK note has the same gap. +- **The upsampling boundaries** (L1145–L1180): the 4:2:0 fixtures take the H = V = 2 path, so a 4:2:2 + image or an odd width would exercise `horizontal == maxH`/`vertical == maxV` and the edge column. The + `x / horizontal` arithmetic survives because the fixtures' chroma is a whole multiple of the luma. +- **The null-table and budget boundaries** (L946, L1055): a stream naming a table it never defined, and an + image exactly at the byte budget. +- **The bit reader's marker branches** (L1264–L1327): the truncation theory reaches some of them; a + stuffed `FF 00` or a marker inside entropy data reaches the rest. The `symbol >>> 4` and `>>> 8` + mutations are equivalent — a symbol comes from a byte and is never negative. +- **The baseline marker and segment edge cases** (L82–L248) are unchanged from the baseline reading. + +None of the survivors is a statement to remove: each is a variant or a boundary the fixtures do not state, +and they are recorded rather than papered over. The two conventions the progressive decoder had to get +right — the sign bit before the correction bits, and 1 meaning positive — are killed by the progressive +fixtures, which is why they are written down in ADR-005. + +**Harness note (5.1.0).** On this workstation, `dotnet stryker` 5.1.0 in solution context mutates the whole +tree however the scope is stated: the `mutate` entry in `stryker-config.json`, `--mutate` and `--since:main` +all leave 23,042 mutants, and the documented 5.1.0 example (`--mutate "**/PdfDiagnoserService.cs"`, 13 +mutants in the 5.1.0 record above) reads the whole tree here. Project mode is refused with "Test projects +cannot be specified when running Stryker in solution context". The reading above is therefore taken with a +local 5.0.0 — `dotnet tool install dotnet-stryker --version 5.0.0 --tool-path %TEMP%\stryker-tools` — run +from the repository root with `--project` and the command-line `--mutate`, which applies the filter (839 of +20,004). The pin, the config and the workflow are unchanged; the version is stated because 5.1.0 rejected +one mutant at compile time in the earlier comparison, so a 5.1.0 re-run may move a point either way. diff --git a/docs/parity.md b/docs/parity.md index f82cfc2..6124dfb 100644 --- a/docs/parity.md +++ b/docs/parity.md @@ -37,7 +37,7 @@ No “Must” row is unowned. The owner is the story that closes it: `US1 creato | 11 | Digital signatures: sign, validate, timestamp, report integrity and trust separately | PDFsharp: yes. PdfPig: no. | Must: detached CMS over a reserved byte range, X.509, optional caller-supplied RFC 3161 token, integrity and trust as separate results. | 1.1 | US4 trust | `PdfDocumentSignerTests.Writes_a_signature_that_verifies_against_its_own_bytes`, `Signing_a_second_time_keeps_the_first_signature_valid`, `Embeds_the_timestamp_token_the_caller_supplies` (T080, T087), `PdfDocumentSignatureValidatorTests.Reports_a_changed_byte_as_a_digest_mismatch`, `Keeps_integrity_independent_of_trust`, `Reports_a_request_for_an_online_check_without_making_one` (T081) | `python docs/tools/validate-signatures.py --trust-signer ` (pyHanko, T093) and `uv run --with pikepdf python docs/tools/validate-pdf.py ` | Met (2026-10-23) on pyHanko and qpdf: a signed revision reports `intact` with coverage `ENTIRE_FILE`, a twice-signed one reports `intact`/`ENTIRE_REVISION` then `intact`/`ENTIRE_FILE`, both `trusted` against the signer's own certificate as anchor, and qpdf `--check` reports no problems on either file. A timestamp token is embedded, never checked: the authority is the caller's. The `sign-and-archive` example (T094) writes a PDF/A-3b file and signs it afterwards — the order that keeps the archival bytes intact — and pyHanko reads what it wrote as field `ArchivalSignature`, `integrity intact (coverage ENTIRE_FILE)`, signed at the moment the options state (2026-10-24) | | 12 | PDF/A-1b, -2b, -2u, -3b, -3u and -4 output | PDFsharp: PDF/A manager. PdfPig: no. | Must: embed every font, write XMP and an output intent, remove encryption and forbidden actions, refuse what cannot conform. | 1.1 | US4 trust | `PdfATests.Writes_the_structures_the_profile_requires`, `Refuses_a_font_it_cannot_embed`, `Refuses_transparency_for_pdf_a_1b_but_writes_it_for_2b` (T082) | `python docs/tools/validate-pdfa.py --verapdf --profile 2b archive-2b.pdf`, and the same for every other profile: veraPDF 1.30.4 reports `PASS` for all six profiles of an authored page and of `SafeWebCore.pdf`, by hand and on every push (`compliance.yml`; T093, T096) | Met — the output intent is sRGB built in, or the caller's own ICC profile and condition, written as supplied and checked as ICC first; PDF/A-3 embedded files are associated through `/AF` with a stated relationship; levels B and U are written, and PDF/A-4 writes the base level of ISO 19005-4; PDF/A-4f/-4u, a merge-time profile and level A are out of scope, and no conformance is claimed without a validator | | 13 | Positioned letters, words and blocks; configurable segmentation and reading order | PDFsharp: not a reading API. PdfPig: yes. | Must: `PdfLetter` with Unicode, size, font name, box and baseline; word and block strategies; raw and column-aware reading order with the strategy in the result. | 1.1 | US3 reading | `PdfLayoutReaderTests.Reads_one_letter_per_glyph_in_content_order`, `Splits_a_two_column_page_into_one_block_per_column`, `Reads_a_page_of_two_columns_left_to_right_whatever_order_the_content_used` (T059–T060), `PdfLayoutCorpusTests.Meets_SC_004_on_the_labelled_corpus`, `PdfLayoutCorpusTests.Table_page_is_still_read_column_by_column` (T076) | `python docs/tools/score-layout.py --labels tests/SharpCorePDF.Tests/Assets/parity/labelled --known-order-mismatch table` (T075) | Met on the labelled corpus (T075–T076): recall and boxes 100% on all five documents, reading order 100% on four. `table` is recorded as a known order mismatch — a table is symmetric under the XY cut, so a cell becomes a block and the order comes out column by column, which is the order a two-column page needs; table semantics are out of contract. `PdfLayoutCorpusTests.Table_page_is_still_read_column_by_column` fails if the limitation ever goes away | -| 14 | Images, hyperlinks, form values, annotations, embedded files, metadata, bookmarks | PDFsharp: partial. PdfPig: yes. | Must: expose each resource through the read side, decode image pixels when asked, and warn on an unresolved destination or unsupported filter. | 1.1 partial, pixels in 1.2 | US3 reading | `PdfLayoutReaderTests.Reads_the_links_of_a_page_and_their_uris`, `Resolves_a_direct_and_a_named_destination`, `Keeps_an_unresolved_destination_and_says_so`, `PdfContentReaderTests.Returns_every_annotation_with_its_geometry_and_text`, `Returns_the_field_tree_with_qualified_names`, `PdfContentReaderTests.Exposes_the_samples_of_a_decoded_image_as_pixels`, `Does_not_expose_a_jpeg_as_pixels`, `Does_not_expose_the_pixels_of_a_filter_it_cannot_decode`, `Exposes_a_palette_image_as_the_colours_of_its_table`, `Exposes_a_palette_whose_table_is_a_string`, `Exposes_bilevel_samples_as_one_byte_per_sample`, `Exposes_four_bit_samples_at_the_scale_of_their_depth`, `Exposes_two_bit_samples_at_the_scale_of_their_depth`, `Exposes_a_cmyk_image_as_four_device_samples_per_pixel`, `Exposes_the_samples_of_an_iccbased_image`, `Does_not_expose_the_samples_of_a_separation_space`, `Does_not_expose_a_palette_image_whose_index_is_outside_its_table`, `Leaves_an_expansion_above_the_budget_out_of_the_record`, `Exposes_the_samples_of_an_image_that_declares_no_filter`, `Keeps_an_expansion_that_fits_the_budget_exactly`, `Exposes_four_bit_samples_of_a_three_channel_image`, `Does_not_expose_an_image_that_states_no_extent`, `Exposes_the_samples_of_a_profile_by_the_components_it_states`, `Does_not_read_a_palette_whose_table_is_empty`, `Refuses_to_resolve_a_colour_space_without_a_document`, `Resolves_the_calibrated_forms_of_the_device_spaces`, `Does_not_build_a_palette_over_a_palette_or_over_a_function`, `Reads_a_palette_whose_declared_index_is_zero`, `Clamps_a_declared_index_to_the_entries_the_table_holds`, `Does_not_read_an_icc_space_that_states_no_profile`, `Exposes_the_pixels_of_an_inline_image_stated_by_an_abbreviation` (T072) | `uv run --with pypdf python -c "from pypdf import PdfReader; ..."` and `validate-pdf.py` on the corpus, plus the `score-layout.py` run of T079 (5 documents, 0 short of a threshold) and an external parse of all four export formats (T079) | Met in 1.2 (2026-10-24): every resource the row names is exposed — images with their payload, geometry, mask and pixels, hyperlinks with resolved and unresolved destinations, form values through `PdfFormFieldInfo`, annotations, embedded files, metadata and bookmarks; pixels are `Gray8`, `Rgb24` or `Cmyk32` for every filter the library decodes and for an image whose stream declares no filter at all — a palette applied so an `Indexed` image yields colours, samples narrower than a byte widened, four device samples left as CMYK rather than converted — and `null` for a filter it does not decode, which is reported with `ImageFilterUnsupported` rather than dropped. `RunLengthDecode` (2026-11-14), `CCITTFaxDecode` (2026-11-15) and baseline `DCTDecode` (2026-11-15) joined the decoded filters and yield pixels; progressive JPEG, `JPXDecode` and `JBIG2Decode` remain pass-through, in the order ADR-005 schedules them | +| 14 | Images, hyperlinks, form values, annotations, embedded files, metadata, bookmarks | PDFsharp: partial. PdfPig: yes. | Must: expose each resource through the read side, decode image pixels when asked, and warn on an unresolved destination or unsupported filter. | 1.1 partial, pixels in 1.2 | US3 reading | `PdfLayoutReaderTests.Reads_the_links_of_a_page_and_their_uris`, `Resolves_a_direct_and_a_named_destination`, `Keeps_an_unresolved_destination_and_says_so`, `PdfContentReaderTests.Returns_every_annotation_with_its_geometry_and_text`, `Returns_the_field_tree_with_qualified_names`, `PdfContentReaderTests.Exposes_the_samples_of_a_decoded_image_as_pixels`, `Does_not_expose_a_jpeg_as_pixels`, `Does_not_expose_the_pixels_of_a_filter_it_cannot_decode`, `Exposes_a_palette_image_as_the_colours_of_its_table`, `Exposes_a_palette_whose_table_is_a_string`, `Exposes_bilevel_samples_as_one_byte_per_sample`, `Exposes_four_bit_samples_at_the_scale_of_their_depth`, `Exposes_two_bit_samples_at_the_scale_of_their_depth`, `Exposes_a_cmyk_image_as_four_device_samples_per_pixel`, `Exposes_the_samples_of_an_iccbased_image`, `Does_not_expose_the_samples_of_a_separation_space`, `Does_not_expose_a_palette_image_whose_index_is_outside_its_table`, `Leaves_an_expansion_above_the_budget_out_of_the_record`, `Exposes_the_samples_of_an_image_that_declares_no_filter`, `Keeps_an_expansion_that_fits_the_budget_exactly`, `Exposes_four_bit_samples_of_a_three_channel_image`, `Does_not_expose_an_image_that_states_no_extent`, `Exposes_the_samples_of_a_profile_by_the_components_it_states`, `Does_not_read_a_palette_whose_table_is_empty`, `Refuses_to_resolve_a_colour_space_without_a_document`, `Resolves_the_calibrated_forms_of_the_device_spaces`, `Does_not_build_a_palette_over_a_palette_or_over_a_function`, `Reads_a_palette_whose_declared_index_is_zero`, `Clamps_a_declared_index_to_the_entries_the_table_holds`, `Does_not_read_an_icc_space_that_states_no_profile`, `Exposes_the_pixels_of_an_inline_image_stated_by_an_abbreviation` (T072) | `uv run --with pypdf python -c "from pypdf import PdfReader; ..."` and `validate-pdf.py` on the corpus, plus the `score-layout.py` run of T079 (5 documents, 0 short of a threshold) and an external parse of all four export formats (T079) | Met in 1.2 (2026-10-24): every resource the row names is exposed — images with their payload, geometry, mask and pixels, hyperlinks with resolved and unresolved destinations, form values through `PdfFormFieldInfo`, annotations, embedded files, metadata and bookmarks; pixels are `Gray8`, `Rgb24` or `Cmyk32` for every filter the library decodes and for an image whose stream declares no filter at all — a palette applied so an `Indexed` image yields colours, samples narrower than a byte widened, four device samples left as CMYK rather than converted — and `null` for a filter it does not decode, which is reported with `ImageFilterUnsupported` rather than dropped. `RunLengthDecode` (2026-11-14), `CCITTFaxDecode` (2026-11-15) and `DCTDecode` (baseline and progressive, 2026-11-15) joined the decoded filters and yield pixels; `JPXDecode` and `JBIG2Decode` remain pass-through, in the order ADR-005 schedules them | | 15 | Public read-only object graph and content-stream inspection | PDFsharp: partial. PdfPig: yes. | Must: immutable `PdfObjectView`/`PdfContentOperationView`, dangling references visible, operation offsets reported. | 1.1 tests, the rest 1.2 (completed 2026-11-06, Unreleased) | US3 reading | `PdfObjectInspectorTests.Returns_one_node_per_listed_object_in_number_order`, `Projects_a_reference_as_a_node_and_never_follows_it`, `Reports_a_dangling_reference_once_for_the_target_it_names`, `Reads_the_operations_of_a_page_in_the_order_it_states_them` | `python docs/tools/validate-pdf.py artifacts/graph.pdf` on the object-graph fixture, run 2026-11-06: qpdf names the file valid with warnings, and the warnings are exactly the two deliberate dangling references to object 99 the fixture states; an independent scan of the same bytes finds objects 1-7 and no object 99, which is what the inspector reports (seven nodes, three unresolved references). The mutation attempt is recorded too: 77.8% over `PdfObjectInspector.cs` | Evidenced (2026-11-06) - twelve tests in `PdfObjectInspectorTests` are in the suite, the external run above agrees with the inspector on which objects exist and which reference dangles, and the mutation score for the inspector is recorded rather than assumed | | 16 | Export to ALTO, PAGE XML, hOCR and SVG text | PDFsharp: no. PdfPig: yes. | Must: four exporters in page space, escaped text, a valid empty-page element, and a result that names the strategy used. | 1.1 exporters, the rest 1.2 (completed 2026-11-06, Unreleased) | US3 reading | `PdfLayoutExportTests.Exports_alto_pagexml_hocr_and_svg`, `Writes_a_page_element_for_a_page_with_no_text`, `Escapes_text_that_looks_like_markup`, `Returns_the_blocks_it_wrote_with_the_strategy_that_produced_them` (T063) `Writes_the_metadata_times_the_page_schema_requires` (2026-11-06) | ALTO 4.2 and PAGE XML 2019-07-15 validated with `lxml` against the published schemas (`alto-4-2.xsd`, `pagecontent-2019-07-15.xsd`), run 2026-11-06: both exports valid after the run found the PAGE document stating a creator and no times - the schema requires `Created` and `LastChange`, which the exporter now writes at a fixed instant so the bytes stay reproducible | Evidenced (2026-11-06) - four exporters and their tests are in the suite, the hOCR and SVG documents are checked by their own tests, and the schema run T079 named is recorded: the ALTO and PAGE exports validate against the published schemas, with the PAGE metadata defect the run exposed now fixed and pinned by a test | | 17 | Rendering pages to images or raster previews | PDFsharp: no. PdfPig: no. | Non-goal: no rasteriser, thumbnail generator, viewer or print pipeline. | 1.1 (excluded) | Product boundary (ADR-001) | None by design | Out-of-scope statement in `README.md`, `docs/manual/src/02-what-it-is-not.md` and ADR-001 | Out of scope | diff --git a/src/SharpCorePDF.Infrastructure/Pdf/JpegDecoder.cs b/src/SharpCorePDF.Infrastructure/Pdf/JpegDecoder.cs index 8987af2..0563c3d 100644 --- a/src/SharpCorePDF.Infrastructure/Pdf/JpegDecoder.cs +++ b/src/SharpCorePDF.Infrastructure/Pdf/JpegDecoder.cs @@ -43,6 +43,13 @@ internal sealed class JpegDecoder private int maxV; private int restartInterval; private int adobeTransform = -1; + private bool progressive; + private JpegComponent[] scanComponents = []; + private int scanStart; + private int scanEnd; + private int scanHigh; + private int scanLow; + private int eobRun; private JpegDecoder(ReadOnlyMemory data) => this.data = data; @@ -148,8 +155,14 @@ private bool Run(int maxBytes, out byte[]? pixels, out string? format, out int w break; case 0xC2: - // Progressive: accepted by ADR-005 after the baseline, not in this slice. - return false; + progressive = true; + + if (!ParseFrame(segment)) + { + return false; + } + + break; case 0xDD: if (segment.Length < 2) { @@ -167,18 +180,36 @@ private bool Run(int maxBytes, out byte[]? pixels, out string? format, out int w return false; } - if (!DecodeScan(offset + length, maxBytes, out pixels, out format, out width, out height)) + if (!progressive) + { + if (!DecodeScan(offset + length, maxBytes, out pixels, out format, out width, out height)) + { + return false; + } + + return format is not null; + } + + if (!DecodeProgressiveScan(offset + length, out var nextOffset)) { return false; } - return format is not null; + offset = nextOffset; + continue; } offset += length; } - return false; + if (!progressive || !ReconstructProgressive() || !BuildOutput(maxBytes, out pixels, out format)) + { + return false; + } + + width = this.width; + height = this.height; + return true; } private static double[] CreateCosines() @@ -350,6 +381,8 @@ private bool ParseScan(ReadOnlySpan segment) return false; } + var named = new JpegComponent[count]; + for (var index = 0; index < count; index++) { var id = segment[1 + (index * 2)]; @@ -363,8 +396,17 @@ private bool ParseScan(ReadOnlySpan segment) component.DcTable = tables >> 4; component.AcTable = tables & 15; + named[index] = component; } + this.scanComponents = named; + + var tail = 1 + (count * 2); + scanStart = segment[tail]; + scanEnd = segment[tail + 1]; + scanHigh = segment[tail + 2] >> 4; + scanLow = segment[tail + 2] & 15; + return true; } @@ -400,6 +442,12 @@ private sealed class JpegComponent public int Stride { get; set; } public int Predictor { get; set; } + + public int[] Coefficients { get; set; } = []; + + public int BlocksX { get; set; } + + public int BlocksY { get; set; } } private bool DecodeScan( @@ -481,6 +529,414 @@ private bool DecodeScan( return BuildOutput(maxBytes, out pixels, out format); } + private bool ReconstructProgressive() + { + var mcuWidth = 8 * maxH; + var mcuHeight = 8 * maxV; + var mcuColumns = (width + mcuWidth - 1) / mcuWidth; + var mcuRows = (height + mcuHeight - 1) / mcuHeight; + var samples = new byte[64]; + + foreach (var component in components) + { + var quant = quantisation[component.Quantisation]; + + if (quant is null) + { + return false; + } + + component.Stride = mcuColumns * 8 * component.H; + component.Plane = new byte[component.Stride * mcuRows * 8 * component.V]; + + for (var blockY = 0; blockY < component.BlocksY; blockY++) + { + for (var blockX = 0; blockX < component.BlocksX; blockX++) + { + var coefficients = component.Coefficients.AsSpan(((blockY * component.BlocksX) + blockX) * 64, 64); + + for (var index = 0; index < 64; index++) + { + coefficients[index] *= quant[index]; + } + + Idct(coefficients, samples); + + var x = blockX * 8; + var y = blockY * 8; + + for (var row = 0; row < 8; row++) + { + samples.AsSpan(row * 8, 8).CopyTo(component.Plane.AsSpan(((y + row) * component.Stride) + x)); + } + } + } + } + + return true; + } + + private bool DecodeProgressiveScan(int dataStart, out int nextOffset) + { + nextOffset = dataStart; + var mcuWidth = 8 * maxH; + var mcuHeight = 8 * maxV; + var mcuColumns = (width + mcuWidth - 1) / mcuWidth; + var mcuRows = (height + mcuHeight - 1) / mcuHeight; + + foreach (var component in components) + { + if (component.Coefficients.Length == 0) + { + component.BlocksX = mcuColumns * component.H; + component.BlocksY = mcuRows * component.V; + component.Coefficients = new int[component.BlocksX * component.BlocksY * 64]; + } + } + + var bits = new JpegBits(data.Span, dataStart); + eobRun = 0; + var mcu = 0; + + if (scanComponents.Length == 1) + { + var component = scanComponents[0]; + var total = component.BlocksX * component.BlocksY; + + for (var blockY = 0; blockY < component.BlocksY; blockY++) + { + for (var blockX = 0; blockX < component.BlocksX; blockX++) + { + if (!DecodeProgressiveBlock(ref bits, component, blockY, blockX)) + { + return false; + } + + mcu++; + + if (restartInterval > 0 && mcu % restartInterval == 0 && mcu < total) + { + if (!bits.TryReadRestart()) + { + return false; + } + + component.Predictor = 0; + eobRun = 0; + } + } + } + } + else + { + var total = mcuColumns * mcuRows; + + for (var mcuY = 0; mcuY < mcuRows; mcuY++) + { + for (var mcuX = 0; mcuX < mcuColumns; mcuX++) + { + foreach (var component in scanComponents) + { + for (var vertical = 0; vertical < component.V; vertical++) + { + for (var horizontal = 0; horizontal < component.H; horizontal++) + { + var blockY = (mcuY * component.V) + vertical; + var blockX = (mcuX * component.H) + horizontal; + + if (!DecodeProgressiveBlock(ref bits, component, blockY, blockX)) + { + return false; + } + } + } + } + + mcu++; + + if (restartInterval > 0 && mcu % restartInterval == 0 && mcu < total) + { + if (!bits.TryReadRestart()) + { + return false; + } + + foreach (var component in scanComponents) + { + component.Predictor = 0; + } + + eobRun = 0; + } + } + } + } + + nextOffset = bits.Position; + return true; + } + + private bool DecodeProgressiveBlock(ref JpegBits bits, JpegComponent component, int blockY, int blockX) + { + var block = component.Coefficients.AsSpan(((blockY * component.BlocksX) + blockX) * 64, 64); + + if (scanStart == 0) + { + if (scanHigh == 0) + { + var dcTable = dcTables[component.DcTable]; + + if (dcTable is null || !dcTable.TryRead(ref bits, out var size) || size > 11) + { + return false; + } + + var difference = 0; + + if (size > 0) + { + if (!bits.ReadBits(size, out var value)) + { + return false; + } + + difference = Extend(value, size); + } + + component.Predictor += difference; + block[0] = component.Predictor * (1 << scanLow); + return true; + } + + // DC refinement: the bit is the next magnitude bit of the coefficient. + if (!bits.TryReadBit(out var refinement)) + { + return false; + } + + if (refinement != 0) + { + block[0] |= 1 << scanLow; + } + + return true; + } + + var acTable = acTables[component.AcTable]; + + if (acTable is null) + { + return false; + } + + if (scanHigh == 0) + { + if (eobRun > 0) + { + eobRun--; + return true; + } + + var index = scanStart; + + while (index <= scanEnd) + { + if (!acTable.TryRead(ref bits, out var symbol)) + { + return false; + } + + var run = symbol >> 4; + var size = symbol & 15; + + if (size == 0) + { + if (run == 15) + { + index += 16; + continue; + } + + eobRun = 1 << run; + + if (run > 0) + { + if (!bits.ReadBits(run, out var extra)) + { + return false; + } + + eobRun += extra; + } + + // The run counts the current block; the rest are the ones that follow it. + eobRun--; + return true; + } + + index += run; + + if (index > scanEnd || !bits.ReadBits(size, out var value)) + { + return false; + } + + block[ZigZag[index]] = Extend(value, size) * (1 << scanLow); + index++; + } + + return true; + } + + // AC refinement: the bits of the coefficients a previous scan stated, plus new ones at this level. + if (eobRun > 0) + { + if (!RefineBand(ref bits, block, scanStart)) + { + return false; + } + + eobRun--; + return true; + } + + var position = scanStart; + + while (position <= scanEnd) + { + if (!acTable.TryRead(ref bits, out var symbol)) + { + return false; + } + + var run = symbol >> 4; + var size = symbol & 15; + + if (size == 0) + { + if (run < 15) + { + eobRun = 1 << run; + + if (run > 0) + { + if (!bits.ReadBits(run, out var extra)) + { + return false; + } + + eobRun += extra; + } + + if (!RefineBand(ref bits, block, position)) + { + return false; + } + + eobRun--; + return true; + } + + // ZRL: sixteen zero coefficients, the nonzero ones refined on the way past. + var remaining = 16; + + while (remaining > 0 && position <= scanEnd) + { + if (block[ZigZag[position]] != 0) + { + if (!Refine(ref bits, block, position)) + { + return false; + } + } + else + { + remaining--; + } + + position++; + } + + continue; + } + + if (size != 1) + { + return false; + } + + // The sign bit precedes the correction bits of the coefficients the run passes over. + if (!bits.TryReadBit(out var sign)) + { + return false; + } + + var newCoefficient = sign != 0 ? 1 << scanLow : -(1 << scanLow); + + // Skip the run's zero coefficients, but refine every nonzero one after them — a nonzero + // coefficient does not consume the run, and its correction bit precedes the new coefficient. + while (position <= scanEnd) + { + if (block[ZigZag[position]] != 0) + { + if (!Refine(ref bits, block, position)) + { + return false; + } + } + else + { + if (run == 0) + { + break; + } + + run--; + } + + position++; + } + + if (position > scanEnd) + { + return false; + } + + block[ZigZag[position]] = newCoefficient; + position++; + } + + return true; + } + + private bool RefineBand(ref JpegBits bits, Span block, int start) + { + for (var index = start; index <= scanEnd; index++) + { + if (block[ZigZag[index]] != 0 && !Refine(ref bits, block, index)) + { + return false; + } + } + + return true; + } + + private bool Refine(ref JpegBits bits, Span block, int index) + { + if (!bits.TryReadBit(out var bit)) + { + return false; + } + + if (bit != 0) + { + block[ZigZag[index]] += block[ZigZag[index]] > 0 ? 1 << scanLow : -(1 << scanLow); + } + + return true; + } + private bool DecodeBlock(ref JpegBits bits, JpegComponent component, int[] coefficients, byte[] samples) { var dcTable = dcTables[component.DcTable]; @@ -555,7 +1011,7 @@ private bool DecodeBlock(ref JpegBits bits, JpegComponent component, int[] coeff private static int Extend(int value, int size) => value < (1 << (size - 1)) ? value - ((1 << size) - 1) : value; - private static void Idct(int[] coefficients, byte[] samples) + private static void Idct(ReadOnlySpan coefficients, byte[] samples) { for (var y = 0; y < 8; y++) { @@ -827,6 +1283,8 @@ public bool ReadBits(int count, out int value) return true; } + public int Position => offset; + public bool TryReadRestart() { bit = 8; diff --git a/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-420-expected.bin b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-420-expected.bin new file mode 100644 index 0000000..45e56b8 Binary files /dev/null and b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-420-expected.bin differ diff --git a/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-420.jpg b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-420.jpg new file mode 100644 index 0000000..a942836 Binary files /dev/null and b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-420.jpg differ diff --git a/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-cmyk-expected.bin b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-cmyk-expected.bin new file mode 100644 index 0000000..a986607 Binary files /dev/null and b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-cmyk-expected.bin differ diff --git a/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-cmyk.jpg b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-cmyk.jpg new file mode 100644 index 0000000..64e5b70 Binary files /dev/null and b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-cmyk.jpg differ diff --git a/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-expected.bin b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-expected.bin new file mode 100644 index 0000000..64dc774 Binary files /dev/null and b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-expected.bin differ diff --git a/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-gray-expected.bin b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-gray-expected.bin new file mode 100644 index 0000000..c23fd77 Binary files /dev/null and b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-gray-expected.bin differ diff --git a/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-gray.jpg b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-gray.jpg new file mode 100644 index 0000000..55bb99b Binary files /dev/null and b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-gray.jpg differ diff --git a/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-restart-expected.bin b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-restart-expected.bin new file mode 100644 index 0000000..c993c08 Binary files /dev/null and b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-restart-expected.bin differ diff --git a/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-restart.jpg b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-restart.jpg new file mode 100644 index 0000000..f431d27 Binary files /dev/null and b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-restart.jpg differ diff --git a/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-sparse-expected.bin b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-sparse-expected.bin new file mode 100644 index 0000000..252a90f Binary files /dev/null and b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-sparse-expected.bin differ diff --git a/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-sparse.jpg b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-sparse.jpg new file mode 100644 index 0000000..d7185a8 Binary files /dev/null and b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-sparse.jpg differ diff --git a/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-wide-expected.bin b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-wide-expected.bin new file mode 100644 index 0000000..1ad6019 Binary files /dev/null and b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-wide-expected.bin differ diff --git a/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-wide.jpg b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-wide.jpg new file mode 100644 index 0000000..93ebe9d Binary files /dev/null and b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive-wide.jpg differ diff --git a/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive.jpg b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive.jpg new file mode 100644 index 0000000..8977725 Binary files /dev/null and b/tests/SharpCorePDF.Tests/Assets/jpeg/jpeg-progressive.jpg differ diff --git a/tests/SharpCorePDF.Tests/PdfJpegFilterTests.cs b/tests/SharpCorePDF.Tests/PdfJpegFilterTests.cs index 79f2ecd..10c77a8 100644 --- a/tests/SharpCorePDF.Tests/PdfJpegFilterTests.cs +++ b/tests/SharpCorePDF.Tests/PdfJpegFilterTests.cs @@ -18,6 +18,20 @@ public sealed class PdfJpegFilterTests private static readonly byte[] CmykExpected = File.ReadAllBytes(Fixture("jpeg-cmyk-expected.bin")); private static readonly byte[] Restart = File.ReadAllBytes(Fixture("jpeg-restart.jpg")); private static readonly byte[] RestartExpected = File.ReadAllBytes(Fixture("jpeg-restart-expected.bin")); + private static readonly byte[] Progressive = File.ReadAllBytes(Fixture("jpeg-progressive.jpg")); + private static readonly byte[] ProgressiveExpected = File.ReadAllBytes(Fixture("jpeg-progressive-expected.bin")); + private static readonly byte[] Progressive420 = File.ReadAllBytes(Fixture("jpeg-progressive-420.jpg")); + private static readonly byte[] Progressive420Expected = File.ReadAllBytes(Fixture("jpeg-progressive-420-expected.bin")); + private static readonly byte[] ProgressiveGray = File.ReadAllBytes(Fixture("jpeg-progressive-gray.jpg")); + private static readonly byte[] ProgressiveGrayExpected = File.ReadAllBytes(Fixture("jpeg-progressive-gray-expected.bin")); + private static readonly byte[] ProgressiveCmyk = File.ReadAllBytes(Fixture("jpeg-progressive-cmyk.jpg")); + private static readonly byte[] ProgressiveCmykExpected = File.ReadAllBytes(Fixture("jpeg-progressive-cmyk-expected.bin")); + private static readonly byte[] ProgressiveWide = File.ReadAllBytes(Fixture("jpeg-progressive-wide.jpg")); + private static readonly byte[] ProgressiveWideExpected = File.ReadAllBytes(Fixture("jpeg-progressive-wide-expected.bin")); + private static readonly byte[] ProgressiveSparse = File.ReadAllBytes(Fixture("jpeg-progressive-sparse.jpg")); + private static readonly byte[] ProgressiveSparseExpected = File.ReadAllBytes(Fixture("jpeg-progressive-sparse-expected.bin")); + private static readonly byte[] ProgressiveRestart = File.ReadAllBytes(Fixture("jpeg-progressive-restart.jpg")); + private static readonly byte[] ProgressiveRestartExpected = File.ReadAllBytes(Fixture("jpeg-progressive-restart-expected.bin")); private static string Fixture(string name) => Path.Combine(TestAssets.Folder, "jpeg", name); @@ -94,6 +108,110 @@ public void Decodes_a_jpeg_with_restart_markers() Assert.InRange(MaxDifference(pixels!, RestartExpected), 0, 2); } + /// A progressive JPEG states its coefficients across several scans and refines them. + [Fact] + public void Decodes_a_progressive_jpeg_within_two_levels() + { + Assert.True(JpegDecoder.TryDecode(Progressive, 1 << 20, out var pixels, out var format, out var width, out var height)); + + Assert.Equal("Rgb24", format); + Assert.Equal(32, width); + Assert.Equal(32, height); + Assert.InRange(MaxDifference(pixels!, ProgressiveExpected), 0, 2); + } + + /// A progressive 4:2:0 JPEG interleaves the DC scan but not the chroma AC scans. + [Fact] + public void Decodes_a_subsampled_progressive_jpeg_within_two_levels() + { + Assert.True(JpegDecoder.TryDecode(Progressive420, 1 << 20, out var pixels, out var format, out var width, out var height)); + + Assert.Equal("Rgb24", format); + Assert.Equal(32, width); + Assert.Equal(32, height); + Assert.InRange(MaxDifference(pixels!, Progressive420Expected), 0, 2); + } + + /// A single-component progressive JPEG refines one plane. + [Fact] + public void Decodes_a_grayscale_progressive_jpeg_within_two_levels() + { + Assert.True(JpegDecoder.TryDecode(ProgressiveGray, 1 << 20, out var pixels, out var format, out var width, out var height)); + + Assert.Equal("Gray8", format); + Assert.Equal(32, width); + Assert.Equal(32, height); + Assert.InRange(MaxDifference(pixels!, ProgressiveGrayExpected), 0, 2); + } + + /// Four components travel through the progressive scans and leave as inverted CMYK. + [Fact] + public void Decodes_a_progressive_cmyk_jpeg_to_four_components() + { + Assert.True(JpegDecoder.TryDecode(ProgressiveCmyk, 1 << 20, out var pixels, out var format, out var width, out var height)); + + var expected = ProgressiveCmykExpected.Select(value => (byte)(255 - value)).ToArray(); + + Assert.Equal("Cmyk32", format); + Assert.Equal(32, width); + Assert.Equal(32, height); + Assert.InRange(MaxDifference(pixels!, expected), 0, 2); + } + + /// A non-square image keeps the column and row counts apart in both grids. + [Fact] + public void Decodes_a_non_square_progressive_jpeg_within_two_levels() + { + Assert.True(JpegDecoder.TryDecode(ProgressiveWide, 1 << 20, out var pixels, out var format, out var width, out var height)); + + Assert.Equal("Rgb24", format); + Assert.Equal(32, width); + Assert.Equal(16, height); + Assert.InRange(MaxDifference(pixels!, ProgressiveWideExpected), 0, 2); + } + + /// Flat areas produce EOB runs and zero runs an AC scan has to skip. + [Fact] + public void Decodes_a_sparse_progressive_jpeg_within_two_levels() + { + Assert.True(JpegDecoder.TryDecode(ProgressiveSparse, 1 << 20, out var pixels, out var format, out var width, out var height)); + + Assert.Equal("Rgb24", format); + Assert.Equal(32, width); + Assert.Equal(32, height); + Assert.InRange(MaxDifference(pixels!, ProgressiveSparseExpected), 0, 2); + } + + /// A restart interval inside a progressive scan resets the predictors and the EOB run. + [Fact] + public void Decodes_a_progressive_jpeg_with_restart_markers() + { + Assert.True(JpegDecoder.TryDecode(ProgressiveRestart, 1 << 20, out var pixels, out var format, out var width, out var height)); + + Assert.Equal("Rgb24", format); + Assert.Equal(32, width); + Assert.Equal(32, height); + Assert.InRange(MaxDifference(pixels!, ProgressiveRestartExpected), 0, 2); + } + + /// A progressive stream that stops inside a scan is refused rather than decoded as far as it got. + [Theory] + [InlineData(50)] + [InlineData(250)] + [InlineData(320)] + [InlineData(390)] + [InlineData(450)] + [InlineData(560)] + [InlineData(620)] + [InlineData(660)] + [InlineData(700)] + public void Refuses_a_truncated_progressive_jpeg(int length) + { + var truncated = Progressive.AsSpan(0, length).ToArray(); + + Assert.False(JpegDecoder.TryDecode(truncated, 1 << 20, out _, out _, out _, out _)); + } + /// A pixel buffer above the budget is measured and nothing is kept. [Fact] public void Measures_a_pixel_buffer_above_the_budget_without_keeping_it()