Repository navigation
ci: check service URLs against the registry - #167
Conversation
Add scripts/check_services.py, which collects the GA4GH service URLs named in the docs and reports services that are not in the Cloud Registry or don't answer service-info. Placeholders and public third-party hosts are skipped; known unregistered services are listed with a reason in service-check.toml. Run it weekly as the job services (Service check) in links.yml, reporting through a shared composite action that the link check now uses too. Closes #159 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Match URLs before a full stop, drop user info and default ports, skip private, link-local and in-cluster hosts, rename the probe, split the check tests, make the report action's labels an input and note cut reports, and document when and how the checks run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Treat h.org. like h.org, and cut long issue reports at whole lines. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewer's GuideAPIAdd the composite issue-reporting action and define its inputs for opening or commenting on a check’s issue.
Add the service-check script, including its command-line options and exit codes, for collecting, normalizing, and checking documented GA4GH service URLs.
Uses of the APIWire the shared report action into the link check and add the scheduled service-check job that runs the script, reports findings, and fails on problems.
ImplementationAdd local-server tests for service URL discovery, registry and service-info fetches, configuration handling, reporting, and command-line behavior.
ConfigurationConfigure hosts to skip and the currently unregistered services that remain allowed while still being probed.
Adjust CI coverage descriptions and include the new service-check module in coverage enforcement.
Update ignore rules for generated service-check output.
DocumentationDocument the weekly service check, its local usage and exit codes, and how to maintain its configuration.
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Linked issuesChecked at commit ef0f4d5 #159 ci: check service URLs against the registry · 3 of 5 met
|
There was a problem hiding this comment.
Hey - I've found 7 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="scripts/check_services.py" line_range="341" />
<code_context>
+ "User-Agent": "elixir-on-cloud-docs-service-check",
+ },
+ )
+ with urllib.request.urlopen(request, timeout=TIMEOUT) as response:
+ body = response.read(MAX_BYTES + 1)
+ break
</code_context>
<issue_to_address>
**Service redirects reach internal hosts**
When an operator of a public service named in the docs makes its `service-info` endpoint redirect to an internal address, `urllib.request.urlopen` follows the redirect without rechecking its destination, so the runner sends a GET to the internal endpoint and accesses a host outside the intended public service targets.
Disable redirects or validate every redirect destination, including its resolved IP address, before following it.
</issue_to_address>
### Comment 2
<location path="scripts/check_services.py" line_range="185" />
<code_context>
+ or address.is_link_local
+ or address.is_unspecified
+ )
+ if "." not in host or any(char in host for char in "%$*"):
+ return True
+ if host.rstrip(".").endswith(IN_CLUSTER_SUFFIXES):
</code_context>
<issue_to_address>
**Loopback URLs get probed**
When a documented service URL uses an abbreviated numeric loopback address such as `127.1`, `ipaddress.ip_address` rejects the address, then the dotted-host check lets it through; the system resolver treats it as loopback, so the checker probes a local service and can report a false failure.
Recognize abbreviated numeric IPv4 loopback addresses as local targets before probing them.
</issue_to_address>
### Comment 3
<location path="scripts/check_services.py" line_range="65" />
<code_context>
+ re.IGNORECASE,
+)
+REGISTRY_URL = re.compile(
+ r"^[ \t]+registry_url:[ \t]*(?P<url>\S+)[ \t]*$", re.MULTILINE
+)
+# Domains reserved for documentation and testing (RFC 2606, RFC 6761), and
</code_context>
<issue_to_address>
**Valid registry settings fail checks**
When `mkdocs.yml` expresses `extra.registry_url` as a quoted scalar, inline mapping, or value followed by a YAML comment, `REGISTRY_URL` does not parse these valid YAML forms, so `default_registry()` returns a value with YAML syntax attached or no value, and `registered_urls` cannot fetch the registry; the check exits 1 instead of checking services.
Parse `mkdocs.yml` with a YAML parser to extract `extra.registry_url` from valid YAML forms.
Also at `scripts/check_services.py:64`.
</issue_to_address>
### Comment 4
<location path=".github/workflows/links.yml" line_range="100" />
<code_context>
- # Show the run as failed, too
- exit 1
+ - name: Report problems
+ if: steps.check.outputs.exit_code == '2'
+ uses: ./.github/actions/report-issue
+ with:
</code_context>
<issue_to_address>
**Resolved service issue stays open**
When a later service check returns 0 after a previous check opened an issue, the `Report problems` step runs only when the check exits 2, and no success path closes the existing issue; users still see resolved findings as active, and a later recurrence comments on the stale issue.
Add a success path that closes the existing issue when the service check reports no problems.
</issue_to_address>
### Comment 5
<location path="scripts/check_services.py" line_range="342" />
<code_context>
+ },
+ )
+ with urllib.request.urlopen(request, timeout=TIMEOUT) as response:
+ body = response.read(MAX_BYTES + 1)
+ break
+ except (OSError, ValueError, http.client.HTTPException) as exc:
</code_context>
<issue_to_address>
**Slow responses stall the check**
When a registry or service sends a response slowly enough to avoid the socket inactivity timeout, `response.read()` has no wall-clock deadline: a server that keeps sending data before each socket timeout can hold the check open until the 15-minute workflow timeout cancels it, before the remaining services are checked or a report is posted.
Enforce a wall-clock deadline for the complete response read, not just a socket inactivity timeout.
</issue_to_address>
### Comment 6
<location path="scripts/check_services.py" line_range="539" />
<code_context>
+ except CheckError as exc:
+ print(f"error: {exc}", file=sys.stderr)
+ return 1
+ findings, checked = check(collect(files), registered, config)
+ text = report(findings, checked)
+ print(text, end="")
</code_context>
<issue_to_address>
**Check failures escape as tracebacks**
When a selected file cannot be read or decoded, or the requested output path cannot be written, `collect()` can raise `OSError` or `UnicodeDecodeError` while reading a selected file, and writing `--output` can raise `OSError`; these operations are outside `main()`’s `CheckError` handler, so the command prints a traceback instead of returning its documented status 1.
Convert these I/O and decoding failures to `CheckError`, or handle them in `main()` and return 1.
Also at `scripts/check_services.py:543-544`.
</issue_to_address>
### Comment 7
<location path=".github/actions/report-issue/action.yml" line_range="53" />
<code_context>
+ --json number,title \
+ --jq 'map(select(.title == env.TITLE)) | .[0].number // empty')
+ if [ -n "$existing" ]; then
+ gh issue comment "$existing" --body-file "$body"
+ else
+ gh issue create --title "$TITLE" --body-file "$body" \
</code_context>
<issue_to_address>
**Reruns post duplicate reports**
When a workflow is rerun after its report step has already commented on the issue, `gh issue comment` posts the report again without deduplicating by workflow run, so maintainers see duplicate comments with the same run link.
Make the report action skip commenting when the issue already contains a report for that workflow run.
Also at `.github/workflows/links.yml:109`.
</issue_to_address>Sourcery assessment
Needs a human reviewer. 7 findings to address first, and if the checker reports a problem incorrectly, the scheduled workflow can create a public issue or post repeated comments, triggering notifications that a revert cannot take back. The issue or comments can be removed, but the notifications and any views of them persist.
Blocking findings: scripts/check_services.py:341, scripts/check_services.py:185, scripts/check_services.py:65, .github/workflows/links.yml:100, scripts/check_services.py:342, and 2 more
Refuse redirects to local hosts, read responses within a deadline, treat numeric shorthands such as 127.1 as local, accept quoted registry_url values and comments, and exit with 1 instead of a traceback when a page or the report file can't be read or written. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # pyproject.toml
Documentation build overview
28 files changed ·
|
Summary
Weekly check that the GA4GH service URLs named in the docs are in the Cloud Registry and answer
service-info.scripts/check_services.py(standard library only): collects URLs with/ga4gh/<api>/v<n>fromREADME.md,docs/andincludes/, normalizes them to the service's base URL (path after the version, user info and default ports dropped;${TES:-https://host}/ga4gh/...shell defaults are understood), and reports each service that is not in the registry or doesn't answerGET <url>/service-infowith a JSON object. The registry isextra.registry_urlofmkdocs.yml(or--registry), counting entries of every environment.example.org/com/net,.test,.invalid,.example,<HOST>,$VAR,localhost,.local,.internal,.svc, names without a dot, loopback, private and link-local addresses) and the public third-party hosts inskip_hostsof the newservice-check.toml(WorkflowHub, Dockstore).service-check.tomlhas an allowlistunregistered, URL → reason, for our services that pages still name but that aren't registered yet. They're still probed forservice-info. The check reports an entry as stale once its service is registered or no page names it any more, so the list only shrinks. Today it holds the three addresses docs: point service pages to the registry #157 will replace (no-auth TESK, proTES, TRS-Filer at Rahti); all three answerservice-info. Against the live registry, the check finds 5 service URLs and no problems.links.yml: new jobservices/Service check(Python 3.13, no dependencies,issues: writeonly, skipped in forks), on the same weekly schedule as the link check. The issue reporting of both jobs moved into a local composite action.github/actions/report-issue(title, intro, report, optional labels; cuts long reports at whole lines below GitHub's body limit, with a note), so the logic isn't duplicated; the link check behaves as before.service-check.toml.tests/unit/test_check_services.pyagainst a local HTTP server (no network), 100 % line and branch coverage;pyproject.tomladds the module to coverage.Closes #159
Verification
python scripts/check_services.py(live registry): "Checked 5 service URLs named in the docs. No problems found.", exit 0.pytest: 260 passed (after mergingmainwith docs: add verified dates and registry steps #166), 100 % coverage (check_names,check_services,registry,last_verified).python scripts/check_names.py,mkdocs build --strict: pass.mypy --stricton the new files: clean.main:gh workflow run links.yml --repo elixir-cloud-aai/docs.Review
Two rounds of adversarial and quality review. Fixed: URLs before a full stop, user info, default ports and trailing dots of hosts, private, link-local and in-cluster hosts skipped, clearer issue intro, truncation note, labels as an action input, probe renamed, focused tests, an unpatched end-to-end test of
main, README on schedule, scope and options.Fixed from the Sourcery review (2fd7e57): redirects to local hosts refused, responses read within a wall-clock deadline (as in
hooks/registry.py), numeric shorthands such as127.1treated as local, quotedregistry_urlvalues and comments accepted, unreadable pages and an unwritable--outputexit with 1 instead of a traceback.Declined review findings
http://[2001:db8::1]/ga4gh/...) are not collected: no page uses one, and a published service has a name.scripts/check_names.py: the scripts stay standalone (different suffixes); ci: flag mentions of archived repositories #160 will reuse this script's helpers instead.links.yml(continue-on-error): the explicit step keeps "couldn't run" (1) apart from "found problems" (2), like the lychee step.service-check.tomlpoint to docs: point service pages to the registry #157 rather than to registration issues: no issue for registering proTES and TRS-Filer exists yet (see docs: point service pages to the registry #157, "Blocked by"); the stale-entry check removes the need to track them here.mkdocs.ymlwith a YAML parser: its!!python/nametags breaksafe_load, and the job has no dependencies; a test guards the setting of this repo. Drop theunregisteredexemption (second review): it is the allowlist that keeps the weekly run meaningful while the registry holds 3 services; listed services are still probed and reported once stale.Open questions
None.
🤖 Generated with Claude Code