Skip to content

ci: check service URLs against the registry - #167

Merged
uniqueg merged 5 commits into
mainfrom
ci/159-service-check
Nov 17, 2026
Merged

uniqueg merged 5 commits into
mainfrom
ci/159-service-check

Conversation

@uniqueg

@uniqueg uniqueg commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Summary

Weekly check that the GA4GH service URLs named in the docs are in the Cloud Registry and answer service-info.

  • New script scripts/check_services.py (standard library only): collects URLs with /ga4gh/<api>/v<n> from README.md, docs/ and includes/, normalizes them to the service's base URL (path after the version, user info and default ports dropped; ${TES:-https://host}/ga4gh/... shell defaults are understood), and reports each service that is not in the registry or doesn't answer GET <url>/service-info with a JSON object. The registry is extra.registry_url of mkdocs.yml (or --registry), counting entries of every environment.
  • Skipped: placeholders and local hosts (example.org/com/net, .test, .invalid, .example, <HOST>, $VAR, localhost, .local, .internal, .svc, names without a dot, loopback, private and link-local addresses) and the public third-party hosts in skip_hosts of the new service-check.toml (WorkflowHub, Dockstore).
  • How it stays green with a registry of 3 services: service-check.toml has an allowlist unregistered, URL → reason, for our services that pages still name but that aren't registered yet. They're still probed for service-info. The check reports an entry as stale once its service is registered or no page names it any more, so the list only shrinks. Today it holds the three addresses docs: point service pages to the registry #157 will replace (no-auth TESK, proTES, TRS-Filer at Rahti); all three answer service-info. Against the live registry, the check finds 5 service URLs and no problems.
  • Exit codes like lychee: 0 fine, 2 problems (opens or comments on one issue "Service URLs in the docs"), 1 the check couldn't run (registry down, invalid config), which fails the run without an issue.
  • links.yml: new job services / Service check (Python 3.13, no dependencies, issues: write only, skipped in forks), on the same weekly schedule as the link check. The issue reporting of both jobs moved into a local composite action .github/actions/report-issue (title, intro, report, optional labels; cuts long reports at whole lines below GitHub's body limit, with a note), so the logic isn't duplicated; the link check behaves as before.
  • README: what the check does, when it runs, how to run it locally, its options and exit codes, and how to maintain service-check.toml.
  • Tests: tests/unit/test_check_services.py against a local HTTP server (no network), 100 % line and branch coverage; pyproject.toml adds the module to coverage.

Closes #159

Verification

  • python scripts/check_services.py (live registry): "Checked 5 service URLs named in the docs. No problems found.", exit 0.
  • pytest: 260 passed (after merging main with docs: add verified dates and registry steps #166), 100 % coverage (check_names, check_services, registry, last_verified).
  • python scripts/check_names.py, mkdocs build --strict: pass.
  • ruff check and format, mypy --strict on the new files: clean.
  • Workflow and action YAML parse; actionlint isn't installed here, so CI and the first scheduled run are the end-to-end test. As with ci: check external links weekly #163, a dispatch run is possible only once the workflow change is on main: gh workflow run links.yml --repo elixir-cloud-aai/docs.
  • CI: all checks pass (PR title, Docs, Test, CI passed, CodeQL, Read the Docs); no open code scanning alerts.

Review

Two rounds of adversarial and quality review. Fixed: URLs before a full stop, user info, default ports and trailing dots of hosts, private, link-local and in-cluster hosts skipped, clearer issue intro, truncation note, labels as an action input, probe renamed, focused tests, an unpatched end-to-end test of main, README on schedule, scope and options.

Fixed from the Sourcery review (2fd7e57): redirects to local hosts refused, responses read within a wall-clock deadline (as in hooks/registry.py), numeric shorthands such as 127.1 treated as local, quoted registry_url values and comments accepted, unreadable pages and an unwritable --output exit with 1 instead of a traceback.

Declined review findings

  • IPv6 literals (http://[2001:db8::1]/ga4gh/...) are not collected: no page uses one, and a published service has a name.
  • Shared file discovery with scripts/check_names.py: the scripts stay standalone (different suffixes); ci: flag mentions of archived repositories #160 will reuse this script's helpers instead.
  • Shorter exit-code handling in links.yml (continue-on-error): the explicit step keeps "couldn't run" (1) apart from "found problems" (2), like the lychee step.
  • Reasons in service-check.toml point to docs: point service pages to the registry #157 rather than to registration issues: no issue for registering proTES and TRS-Filer exists yet (see docs: point service pages to the registry #157, "Blocked by"); the stale-entry check removes the need to track them here.
  • Sourcery: close the issue on a green run, and deduplicate comments on reruns: declined as in ci: check external links weekly #163 (the issue is a to-do closed with the fix; reruns are manual). Parse mkdocs.yml with a YAML parser: its !!python/name tags break safe_load, and the job has no dependencies; a test guards the setting of this repo. Drop the unregistered exemption (second review): it is the allowlist that keeps the weekly run meaningful while the registry holds 3 services; listed services are still probed and reported once stale.

Open questions

None.

🤖 Generated with Claude Code

uniqueg and others added 3 commits November 17, 2026 10:15
Add scripts/check_services.py, which collects the GA4GH service URLs
named in the docs and reports services that are not in the Cloud
Registry or don't answer service-info. Placeholders and public
third-party hosts are skipped; known unregistered services are listed
with a reason in service-check.toml. Run it weekly as the job
services (Service check) in links.yml, reporting through a shared
composite action that the link check now uses too.

Closes #159

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Match URLs before a full stop, drop user info and default ports, skip
private, link-local and in-cluster hosts, rename the probe, split the
check tests, make the report action's labels an input and note cut
reports, and document when and how the checks run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Treat h.org. like h.org, and cut long issue reports at whole lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Reviewer's Guide

API

Add the composite issue-reporting action and define its inputs for opening or commenting on a check’s issue.

  • .github/actions/report-issue/action.yml L1-57

Add the service-check script, including its command-line options and exit codes, for collecting, normalizing, and checking documented GA4GH service URLs.

  • scripts/check_services.py L1-626

Uses of the API

Wire the shared report action into the link check and add the scheduled service-check job that runs the script, reports findings, and fails on problems.

  • .github/workflows/links.yml L1-5, L56-110

Implementation

Add local-server tests for service URL discovery, registry and service-info fetches, configuration handling, reporting, and command-line behavior.

  • tests/unit/test_check_services.py L1-732

Configuration

Configure hosts to skip and the currently unregistered services that remain allowed while still being probed.

  • service-check.toml L1-23

Adjust CI coverage descriptions and include the new service-check module in coverage enforcement.

  • .github/workflows/ci.yml L78
  • pyproject.toml L28-29, L42

Update ignore rules for generated service-check output.

  • .gitignore L25-27

Documentation

Document the weekly service check, its local usage and exit codes, and how to maintain its configuration.

  • README.md L50-82

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@uniqueg
uniqueg marked this pull request as ready for review November 17, 2026 09:33
@sourcery-ai

sourcery-ai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Linked issues

Checked at commit ef0f4d5

#159 ci: check service URLs against the registry · 3 of 5 met

# Requirement Status Notes
1 Collect GA4GH service URLs from the docs and skip placeholders and public third-party hosts ✅ Met The script scans the default documentation paths and skips placeholder, local, and configured third-party hosts.
2 Check each remaining service URL against the registry and service-info 🟡 Partly met The script probes service-info for these URLs, but configured unregistered URLs bypass the registry check.
3 Test the script with 100% coverage ✅ Met check_services is included in coverage settings with a 100% coverage threshold, and the new tests exercise the script.
4 Run a services / Service check job in links.yml with the link check's issue reporting ✅ Met The job runs on the workflow's weekly schedule and uses the shared report-issue action.
5 Have the weekly run report unregistered or unreachable service URLs in one issue 🟡 Partly met Findings are reported through one issue, but listed unregistered services are not reported as unregistered.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 7 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="scripts/check_services.py" line_range="341" />
<code_context>
+                    "User-Agent": "elixir-on-cloud-docs-service-check",
+                },
+            )
+            with urllib.request.urlopen(request, timeout=TIMEOUT) as response:
+                body = response.read(MAX_BYTES + 1)
+            break
</code_context>
<issue_to_address>
**Service redirects reach internal hosts**

When an operator of a public service named in the docs makes its `service-info` endpoint redirect to an internal address, `urllib.request.urlopen` follows the redirect without rechecking its destination, so the runner sends a GET to the internal endpoint and accesses a host outside the intended public service targets.

Disable redirects or validate every redirect destination, including its resolved IP address, before following it.
</issue_to_address>

### Comment 2
<location path="scripts/check_services.py" line_range="185" />
<code_context>
+            or address.is_link_local
+            or address.is_unspecified
+        )
+    if "." not in host or any(char in host for char in "%$*"):
+        return True
+    if host.rstrip(".").endswith(IN_CLUSTER_SUFFIXES):
</code_context>
<issue_to_address>
**Loopback URLs get probed**

When a documented service URL uses an abbreviated numeric loopback address such as `127.1`, `ipaddress.ip_address` rejects the address, then the dotted-host check lets it through; the system resolver treats it as loopback, so the checker probes a local service and can report a false failure.

Recognize abbreviated numeric IPv4 loopback addresses as local targets before probing them.
</issue_to_address>

### Comment 3
<location path="scripts/check_services.py" line_range="65" />
<code_context>
+    re.IGNORECASE,
+)
+REGISTRY_URL = re.compile(
+    r"^[ \t]+registry_url:[ \t]*(?P<url>\S+)[ \t]*$", re.MULTILINE
+)
+# Domains reserved for documentation and testing (RFC 2606, RFC 6761), and
</code_context>
<issue_to_address>
**Valid registry settings fail checks**

When `mkdocs.yml` expresses `extra.registry_url` as a quoted scalar, inline mapping, or value followed by a YAML comment, `REGISTRY_URL` does not parse these valid YAML forms, so `default_registry()` returns a value with YAML syntax attached or no value, and `registered_urls` cannot fetch the registry; the check exits 1 instead of checking services.

Parse `mkdocs.yml` with a YAML parser to extract `extra.registry_url` from valid YAML forms.

Also at `scripts/check_services.py:64`.
</issue_to_address>

### Comment 4
<location path=".github/workflows/links.yml" line_range="100" />
<code_context>
-          # Show the run as failed, too
-          exit 1
+      - name: Report problems
+        if: steps.check.outputs.exit_code == '2'
+        uses: ./.github/actions/report-issue
+        with:
</code_context>
<issue_to_address>
**Resolved service issue stays open**

When a later service check returns 0 after a previous check opened an issue, the `Report problems` step runs only when the check exits 2, and no success path closes the existing issue; users still see resolved findings as active, and a later recurrence comments on the stale issue.

Add a success path that closes the existing issue when the service check reports no problems.
</issue_to_address>

### Comment 5
<location path="scripts/check_services.py" line_range="342" />
<code_context>
+                },
+            )
+            with urllib.request.urlopen(request, timeout=TIMEOUT) as response:
+                body = response.read(MAX_BYTES + 1)
+            break
+        except (OSError, ValueError, http.client.HTTPException) as exc:
</code_context>
<issue_to_address>
**Slow responses stall the check**

When a registry or service sends a response slowly enough to avoid the socket inactivity timeout, `response.read()` has no wall-clock deadline: a server that keeps sending data before each socket timeout can hold the check open until the 15-minute workflow timeout cancels it, before the remaining services are checked or a report is posted.

Enforce a wall-clock deadline for the complete response read, not just a socket inactivity timeout.
</issue_to_address>

### Comment 6
<location path="scripts/check_services.py" line_range="539" />
<code_context>
+    except CheckError as exc:
+        print(f"error: {exc}", file=sys.stderr)
+        return 1
+    findings, checked = check(collect(files), registered, config)
+    text = report(findings, checked)
+    print(text, end="")
</code_context>
<issue_to_address>
**Check failures escape as tracebacks**

When a selected file cannot be read or decoded, or the requested output path cannot be written, `collect()` can raise `OSError` or `UnicodeDecodeError` while reading a selected file, and writing `--output` can raise `OSError`; these operations are outside `main()`’s `CheckError` handler, so the command prints a traceback instead of returning its documented status 1.

Convert these I/O and decoding failures to `CheckError`, or handle them in `main()` and return 1.

Also at `scripts/check_services.py:543-544`.
</issue_to_address>

### Comment 7
<location path=".github/actions/report-issue/action.yml" line_range="53" />
<code_context>
+          --json number,title \
+          --jq 'map(select(.title == env.TITLE)) | .[0].number // empty')
+        if [ -n "$existing" ]; then
+          gh issue comment "$existing" --body-file "$body"
+        else
+          gh issue create --title "$TITLE" --body-file "$body" \
</code_context>
<issue_to_address>
**Reruns post duplicate reports**

When a workflow is rerun after its report step has already commented on the issue, `gh issue comment` posts the report again without deduplicating by workflow run, so maintainers see duplicate comments with the same run link.

Make the report action skip commenting when the issue already contains a report for that workflow run.

Also at `.github/workflows/links.yml:109`.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 7 findings to address first, and if the checker reports a problem incorrectly, the scheduled workflow can create a public issue or post repeated comments, triggering notifications that a revert cannot take back. The issue or comments can be removed, but the notifications and any views of them persist.

Blocking findings: scripts/check_services.py:341, scripts/check_services.py:185, scripts/check_services.py:65, .github/workflows/links.yml:100, scripts/check_services.py:342, and 2 more


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment thread scripts/check_services.py Resolved
Comment thread scripts/check_services.py Resolved
Comment thread scripts/check_services.py Resolved
Comment thread .github/workflows/links.yml Resolved
Comment thread scripts/check_services.py Resolved
Comment thread scripts/check_services.py Resolved
Comment thread .github/actions/report-issue/action.yml Resolved
uniqueg and others added 2 commits November 17, 2026 10:42
Refuse redirects to local hosts, read responses within a deadline,
treat numeric shorthands such as 127.1 as local, accept quoted
registry_url values and comments, and exit with 1 instead of a
traceback when a page or the report file can't be read or written.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New gaps against the linked issues

Comment thread scripts/check_services.py Resolved
@uniqueg
uniqueg merged commit 7c965db into main Nov 17, 2026
21 checks passed
@uniqueg
uniqueg deleted the ci/159-service-check branch November 17, 2026 10:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: check service URLs against the registry

1 participant