Repository navigation
feat(web): v0.8.0 — the anonymous web scan - #65
Merged
Merged
Conversation
The plain-text deliverables (results/repository, results/keyword, results/org, results/user, results/profile) were extensionless: a double-click did nothing and the tool's non-developer audience had to use an 'Open with' dance to read their results. Every text result file now carries a .log extension, so it opens natively in the default editor (Windows Notepad included). The extension is appended inside write_text, after sanitize_file_name: the reserved-device-name prefix, the 200-byte stem cap and the anti-clobber hash stay on the stem, an unchanged target keeps its exact name plus the extension, and a raw name already ending in .log is never stripped (the doubled extension is harmless and pinned). The full path component stays far below the filesystem's 255-byte limit (max stem 200 + hash 17 + extension 4). Doc claims naming the extensionless paths are updated in the same commit (README sample outputs and path rules, doc/src/OUTPUT.md layout table, main.rs rustdoc): a feat commit must never ship a doc/code divergence. The .log file remains a complete snapshot of the run, not an append-only log. --json and --csv paths stay exactly as the user names them.
The book grows from three pages to one page per subject, so the README can shrink to a short landing without losing a single documented claim: every deep-dive README section moves into the page of the feature it describes. - SCANNING.md: scan & search end to end (URL rules with the .git passthrough and paste-artifact guarantees, keyword search, advanced qualifiers, the results paths, scripting notes as bullets); - USER.md, ORG.md, EMAIL.md: the three investigation subcommands, each carrying its own JSON shape and its own API-only deltas (the deltas live next to the feature they constrain, per the doc-code contract); - QUALITY.md: data quality, prevention and limitations, last among the topical pages because it cites the vocabulary of the others; - OUTPUT.md: gains the developer-correlation section (one report spec on one page) and points its proxy-check section at API.md; - API.md: gains 'Network egress: proxies and Tor', verified against the bundled HTTP stack sources (case-sensitive lowercase schemes, socks5h resolves hostnames proxy-side, HTTP_PROXY never used for HTTPS-only traffic, set-but-empty shadowing, REQUEST_METHOD marker): anonymous runs through Tor share the exit's 60-requests-per-hour quota, the startup check proves the proxy at launch and not for the whole run, and no rate-limit avoidance technique is documented; - INDEX.md: retargets the responsible-use pointers at the book pages and gains a one-screen map of the guide; the CI/cron umask recipe moves here from the README. SUMMARY orders usage pages before reference pages. No claim is rewritten in the move: the sections were reviewed against the code in 0.7.0 and only the file names carry the 0.8.0 .log extension already shipped by the previous commit.
The README becomes the short landing the guide now allows: pitch with a trust line, responsible-use anchors (ToS link, personal-data warning, provenance), a six-subcommand table with a when-to-use column, a condensed quickstart (per-OS troubleshooting reduced to hints, the full walkthrough lives in the guide), one measured performance number, six copy-paste examples and the pointers. 706 lines become 200. The trust material a reader must not hunt for stays in the README: the results-hold-personal-data blockquote at the first deliverable, the old-or-spoofed and squash-merge caveats, the token-vs-shell-history warning and the exit-IP advice; everything deep links into the guide. Performance is stated as one measured run with its conditions (v0.8.0 release build, one token, HTTPS egress proxy, 2026-10-26: denoland/deno full default-branch history, 17,397 commits across 174 listing pages, 952 unique email addresses, 2 minutes 26 seconds wall time), the proxy round-trips named, no multi-token number and the honesty contracts next to it. It replaces an 'under a minute' claim that could not be reproduced today. CHANGELOG gains the dated [0.8.0] section with the BREAKING .log entry on top of Changed, following the existing bullet convention.
Four review rounds (five persona lenses on the plan, five oracles, then five reviewers on the actual diff, relaunched until every verdict came back OK or CLEAN) converged on these corrections: - the README's search row claimed every matching repository is scanned: the shipped default is --limit 5 (test-pinned), now stated with the flag, and the reverse-lookup confidence grades (commit-author, repo-owner-weak) are named in the subcommand table — the trust material a reader acts on; - QUALITY.md's squash-merge note stated a false mechanism: the scan filters reject the users.noreply.github.com domain (the squashed commit's author email), not the web-flow noreply@github.com address (which is the committer identity); - the egress documentation: http/https proxy schemes are accepted case-insensitively (only the socks schemes are case-sensitive), the startup check mirrors the stack conservatively and can only over-warn, and a set-but-empty variable falls back instead of always going direct; - the per-OS install walkthrough lives in the guide's Installation section, where the README's pointer claims it is; - three rustdoc comments pointed at README sections that moved to the guide (exit codes, UserReport and OrgReport shapes): they name the guide pages now; - the --csv scope line names scan, search and org, the book's JSON examples carry tool_version 0.8.0, and the CHANGELOG matches the shipped guide titles.
The scan page's browser engine must apply exactly the CLI's rules, so everything rule-shaped leaves the CLI crate for a dependency-free git2mail-core: URL parsing (host allowlist, paste-artifact stripping, the percent-decoded dot-segment defense), the usable-email and noreply discipline, log-safety, file-name sanitization, payload parsing, the variant engine and the correlation report. The CLI keeps I/O, threads and its seven-dependency budget; core carries serde, serde_json, regex and log only, published to crates.io so the wasm crate can bind it. While moving, two contracts tighten: - the user investigation's repositories walk applies the org scan's URL-corruption contract: a listing entry whose owner or repository half is a dot segment, carries a stray separator or query fragment is skipped with a warning instead of being scanned - provenance stamping moves to core so every report shape shares it The extraction is behavior-preserving; the parity vectors introduced with the web bindings pin it from the next commit on.
git2mail-wasm exposes the pure core to the browser: URL normalization, the shared email rule, commit-page extraction and file-name sanitization, string-typed (JSON in, JSON out) with no JS objects imported into Rust. The author name is rendered through the CLI's log-safety rule at the boundary, so a hostile commit can never carry bidi overrides or invisible characters into the page's DOM. The parity vectors live once in core/tests/vectors/ and replay on both sides - the CLI crate and the binding - so CLI/web drift is structurally impossible. Two external dependencies (wasm-bindgen, serde_json), default-features off, publish = false.
A zola site, no backend, no secret, no storage API: a hero, one scan form, and a results page whose sections appear as the walk proceeds - honesty first (attribution warning, live anonymous quota with a relative reset, a trust card ABOVE the copy actions, capped/partial warnings ABOVE the address pills, an identity footer naming the open-source project). The page is a defense stack, not a report viewer: - every funnel command's role line leads the command, and every interpolated value passes a safe charset BEFORE it is quoted (cmd.exe has no single quote; the POSIX idiom stays as depth) - shell-unsafe values are withheld with a note routing to the downloaded file; mailable addresses lead the suggested commands (an inert commit-machine domain is demoted, never removed, with a note explaining what such addresses are) - the mail copy quotes any local part that is not a plain RFC 5322 dot-atom and leaves out addresses whose domain has no valid encoding, announced; the spreadsheet copy neutralizes formula-leading cells and the download carries the CLI-shaped .log with a render-time provenance header (never the edited field) - zero-result, quota-death and page-1-quota states each tell one true story; the scan engine's cold load is narrated and a failed load offers a next step Plain words first throughout: no unexplained walk, heuristic, metadata or CLI; the GitHub traffic graph is described accurately (page views and clones, not API calls); the example pills sit together after a short lead; every action button explains itself on hover.
Playwright against a same-origin loopback: the mock GitHub API and the built site share one origin, so the page's own CSP holds unmodified and the suite stays 100% offline. The webServer rebuilds the site against the loopback origin on every run (a stale build is the one way the pins could lie) and refuses port sharing. Directory URLs resolve to their index page like the real host. The twenty-three scenarios pin the contracts byte-exactly: no auto-run, the quoted-bytes funnel, the withheld-values note, the mailable-first demotion (order only - the data keeps every address), the mail and spreadsheet clipboard bytes including the hostile rows, the provenance header lines and their render-time capture, PARTIAL on a mid-walk and on a page-1 quota death without the false 'exposed no commits' note, the 300-event ceiling, the sanitized hostile author name, the zero- result stories, the tooltips, and the no-storage/CSP defenses.
The Pages workflow builds the wasm bindings (wasm-bindgen-cli pinned to the lockfile's version with a mismatch guard, binaryen's wasm-opt -Oz), the mdbook guide and the zola site, then merges one artifact: the site at the root, the guide under /guide/, the wasm glue inside the site tree - with existence tests for every merged piece. The crates publish workflow keeps trusted publishing (OIDC, the cratesio environment) for the CLI: git2mail-core's first release goes out by hand, since crates.io cannot attach a Trusted Publisher to a name that does not exist yet (RFC 3691 defers pending publishers); the comment in the job records when the two-crate publish returns.
CHANGELOG: dated, the workspace/core split, the web scan's engine parity and honesty contracts, the defense stack, twenty-three headless-browser scenarios. README: the browser paragraph, the roadmap ticked, the terminal sample quoted byte-for-byte (including the correlation report's functional em-dash separator, test-pinned). AGENTS: the per-crate dependency budgets and the harness contract. The guide gets the language sweep's verdicts: walk, squash-merge, default branch and heuristic glossed at first use, allowance unified to quota, the fused OUTPUT paragraph repaired, the API.md negation pile split, the 128-byte --variant-pattern cap documented, the --csv clause for the non-deliverable subcommands, and the index sentence reads whole.
…nned Four coverage gaps from the eight-lane audit, closed: - the bulk partial-report guard is a named, tested decision: an aborted run's report reaches the requested --json only when it carries results or skipped addresses - an empty report must never overwrite a previous good deliverable - the --csv wiring is pinned byte-exactly: the correlation CSV opens with the pinned header row, an empty correlation writes the header alone, and the no-effect notice for the subcommands without a correlation is a named function tested on all three shapes - the exit-code contract (3 = completed but empty, 1 = failure, clap owning 2) is a named map with its own test, matching the guide - the spent-search-quota disjunct gets its mutation-resistant pin: a 403 with the search quota spent and a body that announces nothing still parks (deleting the disjunct retires the token and fails), with the core-endpoint control retiring as the contrast
extiop
force-pushed
the
feat/web-v0.8.0
branch
from
October 26, 2026 20:04
b90c1ef to
3df8a59
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR summary
What does this PR do?
Ship v0.8.0: the anonymous web scan (static GitHub Pages, no backend, no
secret, no storage API) on a new dependency-free pure core crate
(
git2mail-core) bound to the browser viagit2mail-wasm— the CLI'sown rules and honesty contracts, with the CLI/web parity pinned by
shared test vectors. Details in the CHANGELOG
[0.8.0]entry.Related issues:
Closes the web-scan milestone announced in the README roadmap
(ANSSI-FR/MLA issue #138's recommended shape: the GUI in web languages).
Type of change