Skip to content

ci: check external links weekly - #163

Merged
uniqueg merged 7 commits into
mainfrom
ci/152-link-check
Nov 17, 2026
Merged

uniqueg merged 7 commits into
mainfrom
ci/152-link-check

Conversation

@uniqueg

@uniqueg uniqueg commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Summary

  • New workflow Links (.github/workflows/links.yml, job links / Link check): lychee checks the external links of docs/**/*.md and README.md every Monday at 06:00 UTC and on manual dispatch. Not a required check, as external sites fail at random; links between pages stay with mkdocs build --strict.
  • Dead links (lychee exit code 2) open one labelled issue "Dead links in the docs", or comment on the open one; other lychee errors fail the run. Skipped in forks.
  • lychee.toml excludes placeholders, localhost, in-cluster names, login-only pages, Slack invites and GA4GH API base URLs (which serve no page).
  • README.md explains how to run the check locally.
  • Fixes two dead links on researcher pages (Funnel default config moved; Krini has no staging branch, the deployment guide is deployment/README.md on dev).
  • Left for docs: point service pages to the registry #157: https://protes.rahtiapp.fi/ (404) in operators/services/protes.md and researchers/index.md; the first run will report it.

Closes #156

Verification

  • lychee 0.21.0 --config lychee.toml "docs/**/*.md" README.md: 743 links, 426 OK, 315 excluded, 2 errors (the proTES URL above).
  • python scripts/check_names.py, mkdocs build --strict, pytest (25 passed, 100 % coverage): pass.
  • CI: all checks pass (CI passed, CodeQL, Read the Docs).
  • A dispatch run on this branch is not possible: GitHub dispatches only workflows that exist on the default branch (HTTP 404). After the merge, run gh workflow run links.yml --repo elixir-cloud-aai/docs once to confirm; it should open one issue for the proTES URL.

Declined review findings

  • Duplicate permissions at workflow and job level: intended (default read-only, issues: write only for the job).
  • GA4GH API-root exclusion not tied to hosts: intended; API roots return 404 by design. Service URLs get their own check in ci: check service URLs against the registry #159.
  • Sourcery: race between issue lookup and creation, replaced pending runs, duplicate comments on manual reruns, cache (removed on purpose: actions/cache saves only after successful jobs). Reasons in the threads.
  • Fallback to an unlabelled issue if labels are missing: the four labels exist in this repo (org label set).

Open questions

None

🤖 Generated with Claude Code

uniqueg and others added 5 commits November 17, 2026 07:02
Add the Links workflow: lychee checks the external links of the pages
and the README every Monday and opens (or comments on) one issue
"Dead links in the docs" when it finds any. Not a required check, as
external sites fail at random. Links between pages stay with the strict
MkDocs build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Krini has no staging branch; its deployment guide is
deployment/README.md on dev. Funnel moved its default config to
internal/config/examples/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Report only lychee's dead-link exit code and fail on other errors,
truncate the issue body, label new issues, skip forks, save the
cache per run, and anchor the example.org exclusion.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
actions/cache saves only after successful jobs, and the check fails
whenever it finds dead links, so the cache never helped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Reviewer's Guide

Configuration

Configure the weekly lychee workflow and its URL exclusions, with the issue-reporting behavior and repository ignore settings needed to run it safely.

  • .github/workflows/links.yml L1-79
  • .gitignore L24-26
  • lychee.toml L1-30

Documentation

Explain the weekly link check and how to run it locally so contributors can reproduce the workflow.

  • README.md L39-49

Update the researcher pages to point to the relocated Funnel configuration and the current Krini README and deployment guide.

  • docs/researchers/crypt4gh-with-protes.md L86
  • docs/researchers/krini.md L10-11

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@uniqueg
uniqueg marked this pull request as ready for review November 17, 2026 06:25
@sourcery-ai

sourcery-ai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Linked issues

Checked at commit 8345d34

#156 ci: check links weekly · 3 of 7 met · 3 to check by hand

# Requirement Status Notes
1 Add the scheduled and manually triggered Links workflow with the pinned lychee action, checking docs/**/*.md and README.md. ✅ Met The workflow defines the Monday schedule and workflow_dispatch, and checks the requested paths.
2 Configure lychee with cache, retries, accept 429, and exclusions for placeholders and login-only pages. 🟡 Partly met Retries, 429 acceptance, and the requested exclusions are configured, but lychee.toml does not configure a cache.
3 Keep the check non-required and open one labelled issue or comment on the open issue, with issues: write limited to the job. ✅ Met The separate workflow has no required-check wiring; the job has issues: write and reports dead-link results by creating or commenting on the titled issue.
4 Document how to run lychee locally in README.md. ✅ Met README.md gives the install link and local command.
5 Verify that a dispatch run on the branch is green or reports real dead links. 👀 Check by hand (runtime) After the workflow exists on the default branch, run it manually and confirm it completes or reports the known dead links.
6 Verify that a forced dead link opens one issue and a later run comments on that same issue. 👀 Check by hand (runtime) Exercise the workflow with a forced dead link and confirm it creates one issue, then comments on it on a subsequent run.
7 Fix dead links found by the first run or list them in an issue. 👀 Check by hand (external system) Confirm that issue #157 lists the remaining proTES URL that the PR says is left for follow-up.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 6 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path=".github/workflows/links.yml" line_range="42" />
<code_context>
+        with:
+          args: >-
+            --config lychee.toml --no-progress
+            "docs/**/*.md" README.md
+          output: lychee/out.md
+          fail: false
</code_context>
<issue_to_address>
**Shared reference links go unchecked**

When a page uses a shared external reference defined in `includes/references.md`, `lychee` scans the raw Markdown inputs without MkDocs’ `pymdownx.snippets` auto-append, so references defined in `includes/references.md` have no targets in its inputs and broken URLs there are never reported.

Pass `includes/references.md` to lychee or otherwise expand MkDocs snippets before scanning.
</issue_to_address>

### Comment 2
<location path=".github/workflows/links.yml" line_range="26" />
<code_context>
+    # Forks would open issues in their own copy every week
+    if: github.repository_owner == 'elixir-cloud-aai'
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    permissions:
+      contents: read
</code_context>
<issue_to_address>
**Slow checks skip issue reporting**

When enough links stall or time out during a run, lychee keeps retrying slow links before the report step can run, and the 15-minute job deadline kills the workflow first; no issue or comment is then created for that run.

Increase the job timeout or reduce the per-link timeout/retry budget so Lychee can finish and reach the reporting step.

Also at `lychee.toml:6-7`.
</issue_to_address>

### Comment 3
<location path=".github/workflows/links.yml" line_range="70-74" />
<code_context>
+          } > body.md
+          existing=$(gh issue list --state open --search "in:title \"$title\"" \
+            --json number,title \
+            --jq "map(select(.title == \"$title\")) | .[0].number // empty")
+          if [ -n "$existing" ]; then
+            gh issue comment "$existing" --body-file body.md
+          else
+            gh issue create --title "$title" --body-file body.md \
+              --label "type: bug" --label "priority: medium" \
+              --label "workload: hours" --label "status: triage"
</code_context>
<issue_to_address>
**Duplicate open issues are created**

When another actor creates an issue with the same title after the list call returns no match and before issue creation, `gh issue create` opens a second issue instead of reusing the matching open issue, violating the one-open-issue behavior.

Use an issue-creation flow that rechecks for and reuses a matching open issue before creating one.
</issue_to_address>

### Comment 4
<location path=".github/workflows/links.yml" line_range="16" />
<code_context>
+  contents: read
+
+concurrency:
+  group: links
+  cancel-in-progress: false
+
</code_context>
<issue_to_address>
**Queued checks get replaced**

When a second run queues while one run is active and another is already pending, gitHub keeps at most one pending run per concurrency group and replaces it when another run queues. If a run is active, a scheduled check is pending, and another dispatch arrives, the scheduled check never runs; a transient dead link it would have reported can recover before the later check.

Use a queueing mechanism that preserves each invocation, or otherwise ensure a later dispatch cannot replace a pending scheduled check.
</issue_to_address>

### Comment 5
<location path=".github/workflows/links.yml" line_range="72" />
<code_context>
+            --json number,title \
+            --jq "map(select(.title == \"$title\")) | .[0].number // empty")
+          if [ -n "$existing" ]; then
+            gh issue comment "$existing" --body-file body.md
+          else
+            gh issue create --title "$title" --body-file body.md \
</code_context>
<issue_to_address>
**Reruns post duplicate report comments**

When a run that has already posted its report is rerun while the issue remains open, `gh issue comment` posts the same report again, and the step then exits with status 1, so the rerun adds a duplicate comment and notification to the issue.

Make report posting idempotent so a rerun does not post the same report again.
</issue_to_address>

### Comment 6
<location path="lychee.toml" line_range="5-29" />
<code_context>
+# Links between pages are checked by `mkdocs build --strict`; lychee checks
+# the external ones
+
+max_retries = 3
+retry_wait_time = 5
+timeout = 30
+max_concurrency = 16
+
+# 429: rate limited, so the target exists
+accept = ["200..=299", "429"]
+
+# Only web links; mail addresses can't be verified
+scheme = ["https", "http"]
+include_mail = false
+
+exclude = [
+  # Placeholders and local services in examples
+  '^https?://([a-z0-9-]+\.)*example\.(org|com)(:|/|$)',
+  '^https?://localhost',
+  '^https?://127\.0\.0\.1',
+  # In-cluster service names in examples
+  '^http://cloud-registry/',
+  # Base URLs of GA4GH APIs, which serve no page (their endpoints do)
+  '/ga4gh/[a-z-]+/v[0-9]+/?$',
+  # Need a login or reject bots
+  '^https://registry\.elixir-on\.cloud/auth/',
+  '^https://join\.slack\.com/',
+]
</code_context>
<issue_to_address>
**Complete lychee configuration**

The issue asks for caching and exclusions for <HOST> placeholders in addition to retries, 429 acceptance, and other exclusions. The config has no cache setting and does not exclude those placeholders, which occur in docs/researchers/cli.md.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 6 findings to address first, and if the link checker or its configuration falsely reports failures, the workflow can publish an issue or comment containing that report. Reverting stops future runs but does not undo the public issue or comment already sent.

Blocking findings: .github/workflows/links.yml:42, .github/workflows/links.yml:26, .github/workflows/links.yml:74, .github/workflows/links.yml:16, .github/workflows/links.yml:72, and 1 more


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment thread .github/workflows/links.yml Resolved
Comment thread .github/workflows/links.yml Resolved
Comment thread .github/workflows/links.yml Resolved
Comment thread .github/workflows/links.yml Resolved
Comment thread .github/workflows/links.yml Resolved
Comment thread lychee.toml Resolved
Also check includes/ (abbreviations and shared references appended by
pymdownx.snippets), and give the job 30 minutes so slow hosts don't
kill it before the report step. From the Sourcery review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New gaps against the linked issues

Comment thread lychee.toml Resolved
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@uniqueg
uniqueg merged commit 4d58650 into main Nov 17, 2026
17 checks passed
@uniqueg
uniqueg deleted the ci/152-link-check branch November 17, 2026 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: check links weekly

1 participant