Skip to content

ci: validate issue and PR metadata - #528

Open
zjw1111 wants to merge 2 commits into
apache:mainfrom
zjw1111:codex/contribution-metadata-checks
Open

zjw1111 wants to merge 2 commits into
apache:mainfrom
zjw1111:codex/contribution-metadata-checks

Conversation

@zjw1111

@zjw1111 zjw1111 commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator

Purpose

Add contribution checks for Conventional Commits PR titles and required PR description sections. Restrict title types to the 11 documented lowercase types while retaining optional scopes and breaking-change markers.

Check Bug report and Feature issues for required information and search confirmation. When information is missing, post template links and the missing items in one bot reminder; reuse that comment as the issue changes and update it after correction.

Default squash merges to the PR title with an empty commit body, and run the metadata regression suites in the existing script-tests job.

Tests

  • Added PR metadata tests covering all allowed types, lowercase enforcement, optional scopes and breaking markers, malformed titles, missing/duplicate/reordered or empty sections, placeholders, Markdown comments and fences, and AI disclosure markers.
  • Added Issue template tests covering required Bug/Feature fields, search confirmation, optional answers, and code-only reproduction steps. Mocked API tests cover reminder creation and deduplication, updates after edits and corrections, comment pagination, spoofed markers, stale event bodies, ignored closed issues/PRs, JSON requests, and API failures.

API and Format

No public C++ API, storage format, or protocol changes.

Documentation

Document the accepted PR title types in CONTRIBUTING.md and clarify that the PR template's Tests section describes test changes and coverage.

Generative AI tooling

Generated-by: OpenAI Codex (GPT-6)

@SteNicholas SteNicholas left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zjw1111, I left the below three comments for this pull request. PTAL.

Comment on lines +55 to +56
marker = "-->" if in_comment else "<!--"
index = line.find(marker)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preserve comment delimiters inside Markdown code spans. A valid PR whose Purpose contains the text below is rejected:

Handle literal `<!--` markers in metadata.

This search treats the literal opener inside the backticks as an HTML comment, leaves in_comment true, and discards every subsequent required heading. The shared parser also incorrectly reports a missing expectations section for an otherwise valid Bug report when its reproduction contains the same literal. Please recognize code spans before stripping HTML comments, and add regression coverage for literal opening delimiters and complete comment examples inside code.

Comment on lines +90 to +91
if marker:
fence = marker.group(1)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Match opening and closing fences inside list items consistently. This is a valid fenced example in a Markdown list:

- ```cpp
  ReadTable();
  ```

The opening line does not match FENCE_PATTERN because it starts with - , but the closing line does match and is treated here as a new opening fence. Placing this example in Purpose therefore makes the parser discard Tests and every later required heading, rejecting an otherwise complete PR. The same problem affects reproduction examples in Bug reports. Please account for list/container prefixes when tracking fences and add a regression with a list-item opening fence followed by the remaining template sections.

Comment on lines +154 to +157
with open(os.environ["GITHUB_EVENT_PATH"], encoding="utf-8") as source:
event = json.load(source)
pr = event["pull_request"]
value = pr.get("title" if args.check == "title" else "body") or ""

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read current PR metadata when checking a re-run. GITHUB_EVENT_PATH contains the original triggering webhook payload, so re-running an existing workflow after a title/body edit still validates the old values. For example, an initial Fix empty input title fails; after correcting it to fix: handle empty input, re-running the original run still reports the title error. The inverse can report success for metadata that is now invalid, and the concurrency group can cancel the newer edit-triggered run when the older run is replayed. Please fetch the current PR by repository/number, as the Issue checker already does for the current issue body, and add a stale-event regression.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants