Skip to content

CI: resolve branch run by walking HEAD history - #3013

Open
leofang wants to merge 1 commit into
NVIDIA:mainfrom
leofang:lookup-run-id-walk
Open

leofang wants to merge 1 commit into
NVIDIA:mainfrom
leofang:lookup-run-id-walk

Conversation

@leofang

@leofang leofang commented Oct 3, 2026

Copy link
Copy Markdown
Member

Alternative to #2999 for the same underlying issue (#2975), for comparison — not for merging without discussion.

Rewrites branch mode in ci/tools/lookup-run-id to walk the branch's commit history from HEAD and return the first commit whose workflow has a successful run passing the artifact check. The one Actions-API filter used in the hot path is ?head_sha=<SHA>, which is not among the server-side filters community#24626 flags as returning stale slices.

Why this instead of #2999's union-and-cross-check approach:

  • gh run list and the server-side branch= / status= / created= filters all drop out — nothing in the hot path hits a filter 24626 names.
  • No union, multi-source dedup, or headSha == BRANCH_HEAD preference encoding — walk order is the preference.
  • No wall-clock freshness cutoff — a legitimately quiet branch whose last green run is 4 days old is correct, not stale, because that run's SHA is in the branch's commit history.
  • The silent-stale mode that caused the Oct 2 nightly regression is impossible by construction: the top of the walk is HEAD, and the "winner" is the newest green reachable from HEAD.

Call cost in the common case:

  • Active branch, HEAD has a green run: ~5 API calls (workflow-id, branch-head, 1 commits page, 1 runs lookup, 0 or 1 artifact listing). Cross-check branch CI run lookup with workflow-runs REST API #2999 currently does ~27.
  • Quiet branch, N commits back to a green run: ~5+N.
  • Scan bounded by LOOKUP_RUN_ID_COMMIT_SCAN_LIMIT (default 200), errors loudly on exhaustion.

Tag mode is untouched (it uses --commit, not an affected filter, and is low call-volume).

Also removes ci/tools/tests/test_lookup_run_id.py — per discussion on #2975, CI tooling that runs on every PR is exercised by CI itself; the orphaned test suite is removed rather than wired in.

-- Leo's bot

Rewrites branch mode in ci/tools/lookup-run-id to walk the branch's
commit history from HEAD and return the first commit whose workflow
has a successful run passing any required artifact check.

The one Actions-API filter used in the hot path is ?head_sha=<SHA>,
which is not among the server-side filters community/discussions/24626
flags as returning stale slices (branch=, status=, created=). The
cross-check / union / wall-clock-freshness logic from NVIDIA#2976 and NVIDIA#2999
is no longer needed.

Also removes ci/tools/tests/test_lookup_run_id.py; CI tooling that
runs on every PR is exercised by CI itself.

Refs NVIDIA#2975.
@copy-pr-bot

copy-pr-bot Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the CI/CD CI/CD infrastructure label Oct 3, 2026
@leofang leofang self-assigned this Oct 3, 2026
@leofang leofang added bug Something isn't working triage Needs the team's attention P0 High priority - Must do! labels Oct 3, 2026
@leofang

leofang commented Oct 3, 2026

Copy link
Copy Markdown
Member Author

/ok to test b8b2ffe

@leofang
leofang requested review from jpascucci-nv and mdboom October 3, 2026 03:28
@leofang
leofang marked this pull request as ready for review October 3, 2026 03:40
@leofang leofang added this to the cuda.core 1.3.0 milestone Oct 3, 2026
@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

@jpascucci-nv jpascucci-nv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My ai says "My take: PR 3013’s branch-walk design is the better mitigation. I’d prefer adopting that strategy, but keep the tests (JRP I agree we can lose the test) and probably add a local head_branch == REF check in the head_sha results to avoid accidentally accepting a same-SHA run from another branch. (JRP: I guess this could happen if we merge a branch?)"

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working CI/CD CI/CD infrastructure P0 High priority - Must do! triage Needs the team's attention

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants