Repository navigation
Preview an import (spec 025) - #113
Merged
Merged
Conversation
Spec 025 (issue #39): a collector uploads an export file and sees what applying it would do, with every row that can't be applied, writing nothing. Reviewed read-only on Opus and on Fable. Rulings (maintainer, 2026-10-15): decisions 1-7 as proposed (replace, a problem row blocks applying, another currency is refused, only the collection view is applied, json_schemer and the rules move into the app, the upload is never stored, Import collection beside Export collection), decision 11 (lists show their first 100 rows) and decision 12 (50,000 rows in under 30 s). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Plan 025 for previewing an import, reviewed read-only on Opus with its findings applied. ADR 0021 records where the format's rules live in the app: json_schemer and a byte-identical copy of the schema. Rulings (maintainer, 2026-10-15, approving the plan): the memory bar is 500 MB for a 50,000-row preview (spec decision 13); spec 022 isn't edited, and spec 025 records that Import collection supersedes its decision 6's menu order (spec decision 14). Spec 1.5.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ExportFormat::Rules applies v1's schema, through an in-app copy kept byte-identical to docs/, and the five further rules; the suite's ExportFormatRules passes well-formedness and lot identity through to it (ADR 0021). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CollectionImport refuses a file as a whole (decision 10) or gives each row's problems (decision 9) and what applying would do: added, changed, unchanged, held by a problem row, removed (decision 1). Nothing is written.
Deviation: Row#text is written `def text(value) = (value if value.is_a?(String))` — without the parentheses Ruby parses the modifier if as guarding the endless def itself, so loading the class raised NameError (undefined local variable 'value').
Deviation: read_rows leaves duplicate_lot out of the failed pointers it passes to identity_readable? — duplicate_lot is reported at /lots/<i> itself, so counting it made a duplicate row's identity unreadable, and its reason ("as row %d") got a nil row number (TypeError). ExportFormat::Rules finds duplicates with only the schema's pointers failed; the reader now judges identity the same way.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Deviation: the AC-1.2 example creates its three held lots in one loop instead of three create lines — RSpec/ExampleLength (16/15); same lots, same assertions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GET /collection/import_preview/new uploads an export file; POST /collection/import_preview shows what applying it would do, or why it is refused (422). Import collection follows Export collection in the account menu and on More (decision 7). Deviation: the AC-1.8 example builds its file before taking the snapshot — the plan's block called `printing` inside the measured block, so the spec's own setup (a catalog entry, identity, set and printing) showed up as writes; the assertion is unchanged, and a mutation that touches a lot in the controller still fails it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Puma's http_content_length_limit, Thruster's MAX_REQUEST_BODY and kamal-proxy's buffering.max_request_body (decision 6; the maintainer approved the Thruster and kamal-proxy settings on 2026-10-15, spec 1.4.0). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pins the maintainer's ruling of 2026-10-15 (spec 025 decision 13): the per-lot pass gives the same problems, in the same order, as a reference whole-file pass on every example and on crafted variants; a lot checked against $defs/lot gives the whole schema's errors once prefixed; and the root schema never sees a lot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ExportFormat::Rules validated the whole file in one json_schemer call, which holds a result tree for every lot. It now validates the file with lots: [] and then each lot against $defs/lot (ExportFormat::Rules.lot_schemer), prefixing pointers with /lots/<i> and mapping each lot's errors before the next. Root-level errors (a missing key) follow the lots' errors, as in the whole-file pass, so problems come in the same order. Data that isn't an object, or whose lots aren't an array, is still validated whole. ADR 0021 is amended to say so. Ruling (maintainer, 2026-10-15): Option A — header validated with lots: [], then each lot against $defs/lot with pointers prefixed /lots/<i>; whole-file pass grew RSS ~1 GB on 50,000 lots against a 500 MB bar. Runner check (indication only): problems_in on the 50,000-lot resolvable file grows RSS 203 -> 269 MB in 8.1 s; bad-price file 203 -> 273 MB in 8.7 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each acceptance criterion with its evidence, AC-1.10's two measurements (the whole-file pass missing 500 MB, the ruled per-lot pass meeting both bars), AC-3.4 by hand, the screenshots, the deviations from the plan and the documents read. The final bin/ci line is left to fill in. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Records both directions of decision 14: spec 025 supersedes spec 022 decision 6's menu order (docs/specs/README.md, .claude/rules/documentation.md). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mode B (Opus, read-only): SPEC-ALIGNED. Fills in the bin/ci line, the commit list and the diff base after the merge of origin/main, and lists the review's minor findings as follow-ups. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #39.
A collector uploads an export file at
/collection/import_preview/new(account menu and More, after "Export collection") and sees what applying it would do. Nothing is written. The page shows lots added, changed, unchanged, held by a problem row, and removed, with their copies, plus profile notes. Each row that can't be applied gets its own problem with a rule and a place. A file that can't be read at all is refused with a 422, giving the reason. Applying is #40; this preview exposes each row's outcome and the held lot it names, so #40 isn't blocked.docs/specs/025-import-preview/spec.md(v1.5.0, approved)docs/specs/025-import-preview/plan.mdjson_schemerand an in-app copy of the schemadocs/specs/025-import-preview/verification.md, which includes 12 screenshots inscreenshots/Maintainer rulings (2026-10-15)
Replace semantics. A problem row blocks applying. A file in another currency is refused whole. Applying never changes the collector's name or email; it takes
collection_view(decisions 1–4).Where the rules live.
json_schemeris in the app bundle with a copy of the schema kept byte-identical, and the rules moved toExportFormat::Rules(decision 5, ADR 0021).The upload. It is read in the request and never stored. It is judged by its content and refused over 32 MB. Request bodies over 40 MB are refused by Puma, Thruster (
MAX_REQUEST_BODY) and kamal-proxy (max_request_body) (decision 6).Lists. Each shows its first 100 entries, then "and N more" (decision 11).
Performance bars. A 50,000-row preview must finish in under 30 s (decision 12) and grow memory by under 500 MB (decision 13). The first measurement missed the memory bar at about 1 GB. The maintainer ruled the plan's fallback: check the header with
lots: [], then each lot against$defs/lot(f8ca320, 861f076, ADR 0021 amended). Re-measured againstbin/devwith a fresh server for each file:Spec 022 isn't edited. Spec 025 records that it supersedes spec 022 decision 6's menu order, and
docs/specs/README.mdrecords it in both directions (decision 14).Checks
bin/ciat 861f076, the last commit that changes code, passed: 1583 examples, 0 failures, 1 pending (pre-existing), seed 22429. Later commits change documents only, plus a merge oforigin/main, which was also documents only.Deviations from the plan
All are listed in
verification.md:Row#textis parenthesised, andread_rowsleavesduplicate_lotpointers out of itsfailedset.Follow-ups (from the review, not fixed here)
price_paidtoo large for a 64-bit count of cents previews as applicable. Adding a row problem for it belongs with Apply an import #40.distill-feature). Spec 025's areas, data portability and release and deploy, have no area doc yet; Area doc: data portability #108 and Area doc: release and deploy #109 will write them.🤖 Generated with Claude Code