Skip to content

The offline link gate never checks absolute self-links #82

Description

@lesnik512

Follow-up to #66, which adopted lychee --offline as a blocking gate in every non-archived repo
but the two templates.

The gap

--offline is what makes the gate deterministic: it blocks network requests and excludes
external URLs rather than erroring on them. But an absolute
https://github.com/modern-python/<this repo>/blob/main/<path> link pointing back at your own
repo is an external URL by that rule, so it is never checked. A relative link to a deleted file
fails the gate; the same link written absolutely passes forever.

This is not theoretical. The #67 migration deleted planning/ and architecture/ in 21 repos and
found two live instances that the gate would have waved through:

  • httpware — 7 links in docs/ pointing into architecture/, all about to 404.
  • faststream-concurrent-aiokafka — one inside middleware.py's nack guard, pointing at
    planning/decisions/…. It was deliberately absolute so PyPI users without a checkout could open
    it, which makes it user-visible on the package page.

Both are fixed. Neither was caught by the gate; both were caught by a human reading the diff.

Online mode is not the fix

Dropping --offline would check them, but it re-introduces exactly what #66 rejected: network
non-determinism in a blocking gate, plus unauthenticated github.com rate limits in CI. #66's
decision to keep the external check weekly and .github-only still stands.

Proposed fix: --remap, still fully offline

A self-link's target is in the repo, so it needs no network — just resolution against the working
tree. lychee --remap does this. Measured against a synthetic case (one valid self-link, one
broken self-link, one broken relative link, one tag-pinned URL, one cross-repo URL):

--offline                          → 1 error   (broken relative only; broken self-link EXCLUDED)
--offline + --remap '…/(?:blob|tree)/main/(.*) file:///input/$1'
                                   → 2 errors  (both, with the remap shown in the message)

The remap line for a repo would be:

            --remap
            'https://github.com/modern-python/<repo>/(?:blob|tree)/main/(.*) file://$PWD/$1'

Two behaviours worth keeping, both confirmed in that test:

  • Tag-pinned URLs stay excluded. …/blob/0.6.1/planning/releases/0.6.1.md does not match
    blob/main/, so it is left alone — correct, since it deliberately points at a historical tree
    that still resolves. (faststream-concurrent-aiokafkaOffer the link checker to the other repos as a reusable workflow #64 is exactly this case.)
  • Cross-repo links stay external. …/modern-di/blob/main/README.md from another repo is not
    remapped, which is right — it is genuinely someone else's tree.

The alternative, and why it is not enough on its own

Rewriting absolute self-links to relative removes the class of problem, and where possible it is
the better fix. But it cannot cover every case: a URL in a docstring rendered on PyPI has no
checkout to be relative to, which is precisely how the faststream-concurrent-aiokafka instance
arose. So the gate still needs to cover what remains.

One thing to decide

#66 justified copying the gate into each repo rather than sharing a reusable workflow on the
grounds that "the config is 5 static lines with no parameters, so there is nothing to drift." A
remap rule contains the repo name, so that argument weakens: the config becomes parameterised
and a copy-paste error (wrong repo name in the pattern) would silently disable the new check
without failing anything. Worth deciding whether that changes the reusable-workflow call, or
whether a cheap guard is enough — e.g. a test asserting the remap pattern matches the repo's own
origin URL.

Scope

The 25 repos carrying the gate. Also worth a sweep for existing absolute self-links at adoption
time, since this gate has never checked them: grep -rn "github.com/modern-python/<repo>/\(blob\|tree\)/main/".

Activity

  1. added 20 commits that reference this issue on Sep 6, 2026
  2. lesnik512 commented on Sep 6, 2026

    @lesnik512
    MemberAuthor

    Landed in all 26 repos carrying the gate.

    What shipped

                --offline
                --no-progress
                --remap
                'https://github\.com/${{ github.repository }}/(?:blob|tree)/main/(.*) file://${{ github.workspace }}/$1'
                '**/*.md'

    The open question, answered by removing it

    The issue asked whether a remap containing the repo name weakens #66's "the config is 5 static lines with no parameters, so there is nothing to drift" enough to justify a reusable workflow or a guard test.

    Neither, because the pattern needs no repo name. ${{ github.repository }} and ${{ github.workspace }} are substituted by Actions at run time, so the block is byte-identical in every repo and holds no parameter to copy wrong. The failure mode named in the issue — "a copy-paste error (wrong repo name in the pattern) would silently disable the new check" — cannot occur, and #66's justification holds unchanged rather than being weakened.

    Confirmed the hazard was real before dissolving it: with a deliberately wrong repo name in the pattern, self-links revert to EXCLUDED and the gate stays green, exactly as feared.

    The one cost: the interpolated repo name is not regex-escaped, so in this repo the pattern renders …/modern-python/.github/… where . is a wildcard. Over-broad, never under-broad, and inert — no repo name collides.

    Correction to the proposed snippet

    The issue's file://$PWD/$1 does not work. lychee-action is a composite action whose entrypoint runs eval lychee ${ARGS} under set -uo pipefail, so:

    • inside the single quotes the remap pair needs, $PWD stays literal and every self-link errors against file:///…;
    • inside double quotes, $1 is an unbound positional and the step dies before lychee runs.

    ${{ github.workspace }} sidesteps both: Actions substitutes it before the shell sees the string. Verified actions/checkout sets no custom path: in any of the 26 links jobs, so it is the checkout root.

    Verification

    Against lychee 0.24.2 — the version lychee-action@v2 pins — over a clean git archive HEAD export of each repo, asserting three things per repo: the tree is green, an injected broken absolute self-link is caught, and tag-pinned and cross-repo URLs stay excluded.

    0/26 before the change, 26/26 after. Both behaviours the issue asked to preserve hold as described.

    CI confirms the remap actually fires, rather than merely parsing: db-retry's run reports Total 32 / Excluded 27 / Errors 0 — 5 OK. The same tree without the remap gives 4 OK / 28 Excluded. The LICENSE self-link moved from excluded to checked.

    Also checked: fragments (#anchor) and ?plain=1 queries resolve correctly in both directions; tree/ and extensionless targets resolve; all org default branches are main, so the /main/ literal is right; actionlint clean.

    Adoption sweep

    The sweep the issue asked for found 31 pre-existing absolute self-links across the 26 repos — 24 LICENSE badges plus real content links in httpware, modern-di (×3), semvertag (×2) and that-depends. All resolve, so no gate went red on adoption.

    Not covered

    The glob is still '**/*.md', so the faststream-concurrent-aiokafka docstring URL this issue cites as user-visible on PyPI remains unchecked, and widening the glob would not reach it — the URL is split across two adjacent string literals. That, and the 13 ADR paths named in Python source org-wide, are #84.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions