Repository navigation
Allow triggering unstable buildbots with the !buildbot PR comment #419
Description
Activity
An alternative is to promote the buildbot to stable.
- https://buildbot.python.org/all/#/builders/1241 : multiple builds succeeded in a row, I would be ok to mark it as STABLE. We can downgrade it to UNSTABLE if it becomes unstable again.
- https://buildbot.python.org/all/#/builders/1225 most builds are success, same.
Do you want to propose a PR to make them STABLE?
I'm fine with changing these to STABLE, and happy to send a PR to make that change!
I was under the impression that marking a builder as STABLE has broader implications that are not appropriate for an experimental and optional build mode (not sure what my impression was based on, other than my own interpretation of the word "stable" :) )You're correct, Itamar. Stable buildbots are special for release managers, so at least until the PEP is officially approved, I'd wait with marking nogil BBs as stable.
I believe that the limitation to only allow running stable buildbots with "!buildbot" is unnecessary.
Oh wait, I was confused about the definition of "stable" here.
For me, unstable means "is known to fail", and stable "is known to be reliable. Don't we have tiers to decide if a builder blocks a released or not? For example, Tier 3 failures do not block a release: https://peps.python.org/pep-0011/#tier-3
I read the Working with buildbots devguide page again, and it mentions stability in two places:
- In this section it mentions only stable buildbots will post a message to PRs that break them (makes sense, I wouldn't want to extend this to unstable buildbots)
- In the section conveniently named "Stability" it mentions stable buildbots are the ones taken into account when making releases
I agree with @vstinner that the semantics of stability vs tiers is confusing. If I combine the devguide with PEP-11, then only TIER_1 and TIER_2 builders can be STABLE, but in practice I see many TIER_3 and NO_TIER builders that are also considered STABLE.
Maybe further clarification is needed. In the meantime, I propose gh-420 to add the unstable builders only to those that can be manually triggered on PRs (@ambv's suggestion above).
- added a commit that references this issue
on Oct 11, 2023 - added a commit that references this issue
on Oct 27, 2023 - added a commit that references this issue
on Oct 30, 2023 - added a commit that references this issue
on Oct 31, 2023 - added a commit that references this issue
on Nov 1, 2023 - added a commit that references this issue
on Nov 1, 2023 - added a commit that references this issue
on Nov 1, 2023 looks good on test PR python/cpython#111583 !
correct builders were triggered and their status is visible on GitHub checks
Reacted by Victor Stinner
this came up several times recently around testing the NoGIL build on PRs - it's currently not possible to do that using the
!buildbot nogilcomment (example attempt) because the nogil builders are marked as unstable.considering the NoGIL build is "experimental", keeping these builders as "unstable" is probably a good idea, but then we need some way to be able to request these builders on PRs.
after chatting with @ambv, I propose that we extend the
!buildbotcommand to support unstable builders.the mechanics can be discussed in this issue. here are the options I am considering:
!buildbot_with_unstable ...or!buildbot[unstable]or!buildbot ... #with_unstable- bikeshedding welcome)