You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
iOS testbed runner prone to failure when two instances start near the same time #129200
The iOS testbed runner starts an iOS simulator; however, the mechanism that starts the simulator doesn't return the ID of the simulator that was started. The runner needs to know the ID of the simulator that was started so that future commands can be directed at that simulator.
The current implementation of the testbed finds this ID by generating a list of know simulators, and then waiting until another entry appears. This works fine until there are 2 test runs started at near the same time, and the runner is unable to identify which simulator this test run has started. This results in test failures like the following:
note: Run script build phase 'Prepare Python Binary Modules' will be run during every build because the option to run the script phase "Based on dependency analysis" is unchecked. (in target 'iOSTestbed' from project 'iOSTestbed')
note: Run script build phase 'Install Target Specific Python Standard Library' will be run during every build because the option to run the script phase "Based on dependency analysis" is unchecked. (in target 'iOSTestbed' from project 'iOSTestbed')
Found more than one new device: {'5CAA0336-9CE1-4222-BFE3-ADA405F766DE', 'DD108383-685A-4400-BF30-013AA82C4A61'}
make: *** [testios] Error 1
program finished with exit code 2
A re-run of the same test job will usually fix the problem, but it shouldn't happen in the first place.
This almost never happens locally, because users don't normally start to test runs in parallel. However, it's a set of conditions that is reasonably likely on the CI machine, as it's very easy for a PR and a main CI job to start at the same time (most commonly, when you merge a PR and immediately backport to a previous release).
I'm going to re-open this; it's going to be a problem for other users who want to use the testbed in CI configurations, as it doesn't play nicely with pytest-xdist. Parallelized test runs can cause multiple iOS tests to run, triggering this issue.
It would also be desirable to increase the parallelization on the iOS worker as well - the CPU is able to handle the load, it's purely the startup ordering sequence that is an issue.
Activity
freakboy3742 commentedon Jan 22, 2025
python/buildmaster-config#573 should be an immediate workaround to avoid the error in CI.
freakboy3742 commentedon Feb 26, 2025
I'm going to re-open this; it's going to be a problem for other users who want to use the testbed in CI configurations, as it doesn't play nicely with
pytest-xdist. Parallelized test runs can cause multiple iOS tests to run, triggering this issue.It would also be desirable to increase the parallelization on the iOS worker as well - the CPU is able to handle the load, it's purely the startup ordering sequence that is an issue.
gh-129200: Add locking to the iOS testbed startup sequence. (#130564)
pythongh-129200: Add locking to the iOS testbed startup sequence. (py…
[3.13] gh-129200: Add locking to the iOS testbed startup sequence. (G…
Restore use of parallel workers for iOS.
Restore use of parallel workers for iOS. (#581)