Skip to content

ci: list docs pages not verified for 6 months - #170

Merged
uniqueg merged 7 commits into
mainfrom
ci/168-verified-dates
Nov 17, 2026
Merged

uniqueg merged 7 commits into
mainfrom
ci/168-verified-dates

Conversation

@uniqueg

@uniqueg uniqueg commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Summary

Lists the docs pages whose operational facts are due for verification, weekly, in one issue (part of #152).

  • scripts/check_verified.py reads the last_verified front matter of docs/**/*.md (standard library only, like the other checks) and lists the pages whose date is older than 6 months (--months N) and the pages still null. Pages without the key are left out. Exit codes as for the other checks: 0 none due, 2 some due, 1 if it can't run. It rejects values the build rejects, and key forms it can't read without a YAML parser (quoted key, repeated key, value on the next line), so it never misreports a page; it also rejects dates in the future. Tests at 100% coverage.
  • New job verified ("Verified dates") in the weekly Links workflow opens or comments on the issue "Docs pages to verify" via .github/actions/report-issue, labelled type: docs.
  • Unlike the other three checks, pages due don't fail the run: they are a reminder, not a fault, and with 25 pages at null today the run would fail every week and hide real link or service failures. Say if you'd rather have the failure.
  • README: "Last verified dates" says what the job does, how to run it locally and how the issue ends (close it once no page is due; the next finding opens a new one). The workflow paragraph now names all four checks.
  • The comment of the Test job in ci.yml already reads "Tests of the scripts in scripts/ and the hooks in hooks/" (updated by ci: check service URLs against the registry #167), so it needs no change.

The first run will open the issue with the 25 pages that are still null.

Unrelated fixes

Verification

  • pytest: 356 passed, coverage 100%.
  • mkdocs build --strict: passes; python scripts/check_names.py: passes.
  • ruff check and ruff format --check on the new files: clean; mypy --strict on the new script and its tests: clean.
  • python scripts/check_verified.py on main's docs: exit 2, 28 pages with the key, 25 null, none stale.
  • CI: all checks green on the PR (CI passed, Docs, Test, PR title, CodeQL, Read the Docs).

Declined review findings

Two review rounds (adversarial and quality); all medium and low findings fixed except:

  • Make the build hook reject the key forms the script can't read, or share parsing. Declined: the script stays dependency-free like the other weekly checks; it fails loudly on those forms, and the README says which form to write.
  • Sourcery: no front matter without a final newline, and CRLF/mid-line delimiters. Declined: the script uses MkDocs' own YAML_RE and universal newlines, so it finds front matter exactly where MkDocs does (replied in the threads).
  • Explicit (? last_verified) and flow-style keys aren't recognized. Declined as unrealistic in front matter; the docstring says so.
  • Adversarial: docs/researchers/federated-workflows.md still names https://protes.rahtiapp.fi as code. Not a link (lychee skips it), and the notebook there asks for the gateway address, so I left it.

Closes #168
Closes #164

🤖 Generated with Claude Code

uniqueg and others added 6 commits November 17, 2026 12:05
The host answers 404 at its root, so the weekly link check failed
(run 44069505306, issue #164). Name the TES API URL instead, which
the service check covers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scripts/check_verified.py lists the docs pages whose last_verified date
is older than 6 months (or --months) or still null, with tests at 100%
coverage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A verified job in the Links workflow opens or comments on the issue
"Docs pages to verify" with the pages due. The README says so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Read pages with a byte order mark as MkDocs does, and fail on a quoted
or repeated last_verified key and on a value on the next line, which
YAML reads but the script would have misread.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pages due are a reminder and stay due for weeks; failing the run would
hide failures of the other checks. The README says how the issue ends.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An indented comment after last_verified: is still null, and a value
after blank or comment lines is still a value. The README says which
form of the key the weekly check reads; the job comment is clearer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@uniqueg uniqueg self-assigned this Nov 17, 2026
@sourcery-ai

sourcery-ai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Reviewer's Guide

Implementation

Add the dependency-free checker that parses verification dates, identifies due pages, and produces the report and CLI exit status.

  • scripts/check_verified.py L1-287

Configuration

Schedule the verification check in the weekly Links workflow and include it in the configured coverage checks.

  • .github/workflows/links.yml L3-7, L161-205
  • pyproject.toml L42

Documentation

Document the weekly report, local checker command, and issue lifecycle, and correct the proTES references to use its working TES API endpoint.

  • README.md L53-58, L191-208
  • docs/operators/services/protes.md L38
  • docs/researchers/index.md L38-39

Tests

Cover date parsing, due-page selection, reporting, CLI behavior, and error handling for the new checker.

  • tests/unit/test_check_verified.py L1-315

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@read-the-docs-community

read-the-docs-community Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Documentation build overview

📚 elixir-on-cloud-docs | 🛠️ Build #35954795 | 📁 Comparing 321db7f against latest (00539bf)

  🔍 Preview build  

3 files changed
± researchers/index.html
± operators/services/index.html
± operators/services/protes/index.html

@uniqueg
uniqueg marked this pull request as ready for review November 17, 2026 11:30
@sourcery-ai

sourcery-ai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Linked issues

Checked at commit 321db7f

#164 Dead links in the docs · 1 of 1 met

# Requirement Status Notes
1 Fix the reported dead proTES links in the docs. ✅ Met Both reported host-root URLs are replaced with the TES API endpoint in the operator and researcher pages.

#168 ci: list docs pages not verified for 6 months · 4 of 4 met

# Requirement Status Notes
1 Add a tested script that lists pages with null or more-than-six-month-old last_verified dates. ✅ Met scripts/check_verified.py scans Markdown files, and the tests are included in the 100% coverage configuration.
2 Use a weekly job to open or update one issue, “Docs pages to verify,” with the due pages. ✅ Met The Monday Links workflow runs the verified job and reports due pages through the shared issue action.
3 Mention the job in the README’s “Last verified dates” section. ✅ Met The section documents the weekly report, local command, and issue lifecycle.
4 Update the Test job comment in ci.yml to name the tested scripts and hooks. ✅ Met The existing comment already names scripts/ and hooks/; the PR notes that this was updated by #167.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 8 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="scripts/check_verified.py" line_range="54" />
<code_context>
+FRONT_MATTER = re.compile(r"^-{3}[ \t]*\n(.*?\n)(?:\.{3}|-{3})[ \t]*\n", re.DOTALL)
+# The key at the top level, its value up to a comment
+KEY = re.compile(
+    r"^last_verified[ \t]*:(?P<value>[^\n#]*?)[ \t]*(?:#[^\n]*)?$", re.MULTILINE
+)
+# Forms of the key that YAML reads but this script doesn't: a quoted key, and
</code_context>
<issue_to_address>
**Malformed dates disappear from reports**

When a `last_verified` value has `#` immediately after the date, without separating whitespace, `KEY` treats the adjacent `#` as a comment, so `read_page` accepts the malformed value as a fresh date and omits the page from the due report.

Treat `#` as a comment only when YAML whitespace separates it from the value, and reject values with an adjacent `#`.
</issue_to_address>

### Comment 2
<location path="scripts/check_verified.py" line_range="51" />
<code_context>
+)
+
+# As MkDocs reads front matter (`mkdocs.utils.meta.YAML_RE`)
+FRONT_MATTER = re.compile(r"^-{3}[ \t]*\n(.*?\n)(?:\.{3}|-{3})[ \t]*\n", re.DOTALL)
+# The key at the top level, its value up to a comment
+KEY = re.compile(
</code_context>
<issue_to_address>
**Front matter pages disappear**

When a page has CRLF front matter, or valid metadata before `last_verified` contains `---` or `...` followed by whitespace and a newline, `FRONT_MATTER` requires LF line endings and accepts a closing delimiter without requiring it to start a line. `read_page` therefore returns `None` for CRLF front matter or for valid metadata containing `---` or `...` mid-line before `last_verified`, so due pages are omitted from the report.

Match front-matter delimiters as complete lines while accepting CRLF line endings.
</issue_to_address>

### Comment 3
<location path="scripts/check_verified.py" line_range="54" />
<code_context>
+FRONT_MATTER = re.compile(r"^-{3}[ \t]*\n(.*?\n)(?:\.{3}|-{3})[ \t]*\n", re.DOTALL)
+# The key at the top level, its value up to a comment
+KEY = re.compile(
+    r"^last_verified[ \t]*:(?P<value>[^\n#]*?)[ \t]*(?:#[^\n]*)?$", re.MULTILINE
+)
+# Forms of the key that YAML reads but this script doesn't: a quoted key, and
</code_context>
<issue_to_address>
**Indented metadata is omitted**

When a page uses valid front matter with an indented root-level `last_verified` key, `KEY` only matches at column zero, so `read_page` returns `None` and stale or unverified pages are omitted from the weekly report.

Make `KEY` recognize valid indented root-level `last_verified` keys.
</issue_to_address>

### Comment 4
<location path="scripts/check_verified.py" line_range="52" />
<code_context>
+
+# As MkDocs reads front matter (`mkdocs.utils.meta.YAML_RE`)
+FRONT_MATTER = re.compile(r"^-{3}[ \t]*\n(.*?\n)(?:\.{3}|-{3})[ \t]*\n", re.DOTALL)
+# The key at the top level, its value up to a comment
+KEY = re.compile(
+    r"^last_verified[ \t]*:(?P<value>[^\n#]*?)[ \t]*(?:#[^\n]*)?$", re.MULTILINE
</code_context>
<issue_to_address>
**Pages without a final newline are skipped**

When a Markdown file's closing front-matter delimiter is the final bytes of the file, with no trailing newline, `FRONT_MATTER` requires a newline after the closing delimiter, so it returns no match when a Markdown file ends immediately after `---` or `...`. MkDocs' front-matter parser accepts that ending, so a stale or unverified page is omitted from the weekly report.

Allow end-of-file as well as a newline after the closing front-matter delimiter.
</issue_to_address>

### Comment 5
<location path="scripts/check_verified.py" line_range="152-153" />
<code_context>
+        not verified yet, by path.
+    """
+    stale = sorted(
+        (p for p in pages if p.verified is not None and p.verified < cutoff),
+        key=lambda p: (p.verified, p.path),
+    )
+    unverified = sorted((p for p in pages if p.verified is None), key=lambda p: p.path)
</code_context>
<issue_to_address>
**Future dates suppress reminders**

When a valid but future date is entered as `last_verified`, `due` only selects dates before the cutoff, so the page is treated as verified and omitted from the report until the cutoff passes its future date plus the configured age.

Reject dates later than the effective check date, including the `--today` value, as malformed.
</issue_to_address>

### Comment 6
<location path="scripts/check_verified.py" line_range="257" />
<code_context>
+    cutoff = months_before(today, options.months)
+    try:
+        files = find_files(options.paths or DEFAULT_PATHS, SUFFIXES)
+        pages = [page for path in files if (page := read_page(path)) is not None]
+    except CheckError as exc:
+        print(f"error: {exc}", file=sys.stderr)
</code_context>
<issue_to_address>
**Overlapping paths inflate counts**

When the caller supplies overlapping paths, such as a directory and a Markdown file inside it, `find_files` returns the same file for each matching argument, so `main` reads duplicate pages and lists them multiple times, inflating the report counts.

Deduplicate the files returned by `find_files` before reading pages.
</issue_to_address>

### Comment 7
<location path="scripts/check_verified.py" line_range="257-258" />
<code_context>
+    cutoff = months_before(today, options.months)
+    try:
+        files = find_files(options.paths or DEFAULT_PATHS, SUFFIXES)
+        pages = [page for path in files if (page := read_page(path)) is not None]
+    except CheckError as exc:
+        print(f"error: {exc}", file=sys.stderr)
+        return 1
</code_context>
<issue_to_address>
**I/O errors escape CLI handling**

When a selected file cannot be read or decoded, or the requested output cannot be created or written, `read_page` and the output-file writes can raise `OSError` or `UnicodeDecodeError`, but `main` catches only `CheckError`; callers get an uncaught exception and traceback instead of the check’s documented error handling.

Catch `OSError` and `UnicodeDecodeError` for input reads and `OSError` for output writes, report the error, and return status 1.

Also at `scripts/check_verified.py:265-266`.
</issue_to_address>

### Comment 8
<location path="scripts/check_verified.py" line_range="173" />
<code_context>
+    Returns:
+        The report.
+    """
+    summary = f"{checked} pages have a `last_verified` date."
+    if not stale and not unverified:
+        return f"{summary} All of them were verified on or after {cutoff}.\n"
</code_context>
<issue_to_address>
**Dated page count is inflated**

When one or more pages have `last_verified` set to null, `~`, or an empty value, `main` includes pages with `verified is None` in `checked`, so the summary says they have a date even though they are listed as unverified; readers see an inflated dated-page count.

Count only pages with a non-`None` `verified` value in the dated-page summary.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 7 findings to address first, and if the date check reports pages incorrectly, the workflow may open an issue or add comments that remain after the change is reverted. Those reports are bounded and can be closed or cleaned up; reverting stops future reports.

Blocking findings: scripts/check_verified.py:54, scripts/check_verified.py:51, scripts/check_verified.py:54, scripts/check_verified.py:52, scripts/check_verified.py:153, and 2 more


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment thread scripts/check_verified.py Resolved
Comment thread scripts/check_verified.py Resolved
Comment thread scripts/check_verified.py Resolved
Comment thread scripts/check_verified.py Resolved
Comment thread scripts/check_verified.py Resolved
Comment thread scripts/check_verified.py Resolved
Comment thread scripts/check_verified.py Resolved
Comment thread scripts/check_verified.py Resolved
Reject future dates, indented keys and a # right after the value, read
each file once, turn read errors into exit 1, and count keys rather
than dates in the summary.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@uniqueg
uniqueg merged commit f1d91c7 into main Nov 17, 2026
17 checks passed
@uniqueg
uniqueg deleted the ci/168-verified-dates branch November 17, 2026 11:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: list docs pages not verified for 6 months Dead links in the docs

1 participant