Repository navigation
RFC: update Pull Requests with summary of CI failures #32306
Description
ncu-ci is quite useful to get summary of failures in our CI, but it still requires us to use an external program to fetch the information from Jenkins. We could use our github-bot to post the information generated by ncu-ci back to the PR, either as a new comment or by editing the bot's comment on that PR. This would also allow us to keep a history of CI failures (since our Jenkins jobs are not persistent).
Is that something other collaborators want as well? I'm looking for feedback before implementing it.
If folks want example of what this would look like, I've updated all CI runs with their respective summaries in #32116. It would look like this:
Or, as a standalone comment:
Activity
sam-github commentedon Mar 16, 2020
My thoughts, FWTW:
On the one hand:
On the other hand:
<details>...</details>guards around the ncu-ci output.As long as it doesn't overwhelm the comment history, I wouldn't block it, though when something fails, I personally haven't really had tons of trouble clicking through to find what the error was, even though it can be tedious.
mmarchini commentedon Mar 16, 2020
Thanks for the feedback!
Agreed. In the examples above I used
detailsto address this issue.That's mostly a Jenkins problem IMO, but I don't think there's any other CI compatible with all platforms we support.
If the bot edits its own comment instead of creating new ones, this shouldn't be an issue.
For me it depends on how many platforms failed. If it's more than two it usually takes way too much time for a task that should be simple. If
node-test-commitandnode-test-pull-requestaggregated result from subprocesses, the situation would be a lot better, but still wouldn't solve the "preserve failures" issue.sam-github commentedon Mar 16, 2020
I don't find Travis much better, just differently annoying.
That might leave the CI result comment floating in the middle of the conversation, instead of where it should be, at the bottom, near the CI results.
Many people fix one problem, and re-CI, I think, particularly after checking two failed platforms and seeing the same error. Even with ncu-ci output, we still have to click through to every build, and then to the console output (usually) to see the problems.
It might be better to spend time better advertising ncu-ci, particularly in onboarding, and getting it to the point where everyone prefers running it to clicking through to Jenkins. Its easy to sell automating a process that everyone already does.
And again, no objections from me. Maybe if everyone sees ncu-ci output, it'll even push folks to help improve the tool.
mmarchini commentedon Mar 16, 2020
I see this as a problem in our process today. Jenkins is too cumbersome to check for failures on every platform, leading people to ignore some platforms before rerunning CI, which can hide flaky tests.
Unless it is a build failure, there's no need to click on the console output. Expanding
<details>also takes way less time than loading the whole Jenkins output.github-actions commentedon Jun 27, 2026
This issue has been marked as stale due to 210 days of inactivity.
It will be automatically closed in 30 days if no further activity occurs. If this is still relevant, please leave a comment or update it to keep it open.
github-actions commentedon Jul 28, 2026
This issue has been automatically closed after 30 days of inactivity following its stale status (no activity for a total of 120 days).
If this is still relevant, feel free to reopen it or leave a comment with additional details so we can continue the discussion.