End-to-end tests that delete a temporary clone fail intermittently on newer git #263

This was generated by AI during triage.

What happened

The test check failed once on PR #258 (run 36844055894) in fails_when_the_base_branch_keeps_moving_during_the_ci_wait, an end-to-end test the branch had not touched. The same code passed on the next push.

The test's on_ci_read hook moves the Base branch from a temporary clone and then deletes the clone. The delete failed:

thirdshift: gh api repos/acme/widgets/commits/40bc65e…/check-runs?per_page=100 failed: rm: cannot remove '/tmp/.tmp6Q3AHI/tmp/tmp.iISOgzJuQ9/.git/objects': Directory not empty
fake: panicked at tests/fakes/gh.rs:1381:5:
the on_ci_read script failed

So the fake gh panicked and the Run under test failed for the wrong reason. That failure then cost a real Run: see #262.

Likely cause (not reproduced)

Something was still writing into the clone's .git/objects while rm -rf walked it. The likely writer is git's auto maintenance: the CI runner has git 2.55, and newer git detaches auto maintenance into the background after a commit, where it holds a lock file in the object directory and may write there after the command has returned.

This is inferred from the error and the version difference, not reproduced. The development machine has git 2.43, which does not detach maintenance this way, and the suite passes there every time. Confirming the cause needs a git of 2.47 or later.

Agent Brief

Category: bug
Summary: Scripts in the end-to-end tests that delete a temporary clone must not race background git maintenance.

Current behavior:
Several end-to-end test files have a script that makes a temporary clone with mktemp -d, commits and pushes from it, and deletes it with rm -rf. The scripts run under bash -e with the scenario's HOME, so they read the gitconfig the test harness writes for each scenario. That gitconfig sets the identity, the default branch and the insteadOf rule, and leaves auto maintenance on. On a newer git the rm -rf intermittently fails with Directory not empty on .git/objects, the script fails, and the fake that ran it panics.

Desired behavior:
No git command run inside a scenario (by thirdshift, a fake, or a script) starts background maintenance, so nothing writes into a repository after the command that touched it has returned. The simplest way is maintenance.auto = false in the gitconfig the harness writes for every scenario, which covers every script at once; gc.auto = 0 does the same for a git too old to know maintenance.auto.

If the agent can confirm the cause on git 2.47 or later (for example by running one of these scripts in a loop with and without the setting), it should say so in the PR. If it cannot, the PR should say the cause is still inferred.

Key interfaces:

• The scenario's gitconfig, written by the test harness (write_gitconfig): gains the setting, with a comment saying why.
• The scripts that clone to a mktemp -d directory and rm -rf it (base_moves_on and its copies across the end-to-end tests): unchanged if the gitconfig setting covers them.

Acceptance criteria:

• git config --get maintenance.auto, run with a scenario's HOME, prints false.
• A comment where the setting is written explains the race it prevents.
• cargo test passes, with no test weakened, skipped or retried to get there.
• The PR states whether the cause was confirmed on git 2.47 or later, or is still inferred.

Out of scope:

• Any change to how thirdshift itself runs git, or to its behaviour outside the tests.
• Retrying, ignoring or deferring the rm -rf in the scripts as a substitute for stopping the background writer.
• Folding the copies of the clone-and-push script into one helper.
• Re-running failed checks on a Declined CI fix: that is Re-run failed checks once before a Declined CI fix fails the Run #262.