Skip to content

Keep Passes clear of live Experiments - #1982

Merged
JacobStephens2 merged 18 commits into
mainfrom
issue-1494
Nov 11, 2026
Merged

JacobStephens2 merged 18 commits into
mainfrom
issue-1494

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Nov 10, 2026 •

Copy link
Copy Markdown
Owner

Summary

Keep factory work from competing with Experiment Trials on the same machine and OS account.

Pass → live Experiment → skip, naming it in the Activity log
Targeted Command → live Experiment → warn and run
Experiment → publish live record → drain Commands → Trials
  --no-wait → record competing Command count → Trials

Waiting Experiments take turns. Stop and resume preserve the normal interrupted lifecycle. Status and the Console name the remaining Commands, and Experiment results format 11 shows the bypass count beside the Verdict.

Closes #1494

Evidence

Execution evidence

Recovery validation — current head 6c2ce8ef6fd8570472300ff31e45d2481c6484eb: Lima Fix run fix-20261111T125448-0500 completed with outcome ready for review on 2026-11-11. It reconciled the branch with main, completed two Repair sessions, and verified the PR was open, ready and mergeable. CI passed the Rust test, Console and site jobs on this head. Release passed, including both Linux artifact builds. Jobs excluded by the workflows' gates were skipped. Self-merge was disabled.

The detailed implementation checks below were recorded before this recovery:

  • Before: cargo test --test experiments host:: -- --nocapture failed with thirdshift: --no-wait requires a value.
    After: the same command passes all six host coordination regressions, covering all seven Pass kinds, manual Ticket and Custom process warnings, waiting and refreshed blockers, Stop/resume, competing Experiments and bypass counts.
  • Before: cargo test --test process a_process_with_an_issue_fills_a_slot_so_a_pickup_run_runs_beside_it_only_above_a_command_limit_of_1 -- --exact --nocapture failed: Pickup returned https://github.com/acme/widgets/pull/1 at a Command limit of one.
    After: the same command passes. Both other behavioral review reproductions were also run exactly as supplied: the waiting-Experiment cycle and omitted text-status blocker failed before their fixes and passed afterward. Their tests are retained.
  • cargo clippy --all-targets -- -D warnings passes.
  • CI exposed an Activity-log compatibility regression. Before: cargo test --test activity_log an_activity_log_that_cannot_be_written_is_one_warning_and_changes_nothing_else -- --exact --nocapture failed with two warnings instead of one. After: the same test passes; admission now reads Command records without depending on Activity-log contents. The entire Activity log suite also passes (16 tests, one ignored fixture entry).
  • CI corrections retain the existing help and platform contracts. Before: cargo test --test command_surface found 71 logical help lines against the 68-line limit; cargo test --test packaging non_linux_compilation_reports_only_the_platform_error -- --exact --nocapture found four diagnostics instead of the one unsupported-platform message. After: the Command surface suite passes (20 tests, one ignored fixture entry; help is 68 lines), and the exact packaging regression passes. The lock module now uses the same Linux gate as surrounding modules.
  • Local final selection: RUST_TEST_THREADS=3 CARGO_BUILD_JOBS=3 /usr/bin/time -p scripts/affected-tests refs/remotes/origin/main, with the documented Chromium setup and a stable THIRDSHIFT_BINARY snapshot. Site passed (94.41s), locked build passed (114.63s, compilation included), then Console finished 186/187 with a five-second timeout in the existing phone Dismiss case. Its exact focused rerun passed: node --test --test-name-pattern='^Inbox row Dismiss works in ready for review with JavaScript on a phone$' console/tests/inbox.test.cjs. A prior local Sections assertion also passed its exact focused rerun. The complete Console CI suite passed on implementation head 345a2aff before recovery.
  • Earlier implementation head 345a2aff: CI and Release pass. cargo nextest run passed all 4,181 tests (97 skipped): 1,216.160s execution, with test compilation included in the 1,284s step. Formatting, clippy, generator checks, and the complete site and Console suites pass. These exact-head CI results supply full-suite validation; the local selector stopped at the browser timeout described above.

Visual evidence

These captures document the implementation before recovery. Before captures use main at 0009a536; After captures use the then-reconciled branch. Every pair uses the same viewport and fixture state. Media is stored in a secret Gist and kept out of repository branches.

Experiment waiting for a Command

Desktop · 1440×900

Before
After

Mobile · 390×844

Before
After

Competing Command count beside the Verdict

Desktop · 1440×900

Before
After

Mobile · 390×844

Before
After

Waiting Command names in the Inbox

Desktop · 1440×900

Before
After

Mobile · 390×844

Before
After

Experiment instructions in the manual

Desktop · 1440×900

Before
After

Mobile · 390×844

Before
After

Experiments on the homepage

Desktop · 1440×900

Before
After

Mobile · 390×844

Before
After

Experiment scheduling in the guide

Desktop · 1440×900

Before
After

Mobile · 390×844

Before
After

Experiment scheduling in the architecture page

Desktop · 1440×900

Before
After

Mobile · 390×844

Before
After

Test seams

  • CLI with fake GitHub and Harness adapters, real advisory locks and owned fixture processes.
  • Retained Command/Experiment records projected into JSON and text status.
  • Console pages and Inbox in Chromium at desktop and mobile widths, including escaping and overflow checks.

Review

The review coverage below was recorded through implementation head 345a2aff. Recovery validation for current head 6c2ce8e is listed above; a fresh review of the recovery changes remains part of PR review.

Reviewed against refs/remotes/origin/main, with the reconciled merge-base pinned to 0009a536. Standards raised one finding; Spec raised three. All four are addressed. Supplemental reviews cover the CI corrections, fresh final reviews cover the reconciled head 20aa5bb2, and both axes inspected the platform correction in implementation head 345a2aff; no new findings were raised. Combined coverage includes all 33 changed files and generated HTML. Changed files left unread: none. Generated-output exclusions: none.

Unaddressed findings

Standards

None.

Spec

None.

Security

None.

Security review outcome

No unaddressed introduced findings.

Merge Danger

Door: two-way

Existing records remain readable through defaults. Consumers of the versioned Experiment JSON need format 11 support.

Blast Radius: scheduling

All Pass kinds now skip during a live Experiment on this account and machine. Already running Commands continue; targeted Commands can still compete after a warning.

Built with claude 2.1.314 · opus · default effort

@JacobStephens2 JacobStephens2 added in-progress A Claim: thirdshift has taken this issue and removed in-progress A Claim: thirdshift has taken this issue labels Nov 10, 2026
@JacobStephens2
JacobStephens2 marked this pull request as draft November 11, 2026 01:08
@JacobStephens2 JacobStephens2 removed the in-progress A Claim: thirdshift has taken this issue label Nov 11, 2026
2026-11-11T01:08:35Z, host orchestrate.etadventures.com. Uncommitted work at the time of failure is included in this commit.
@JacobStephens2 JacobStephens2 added the in-progress A Claim: thirdshift has taken this issue label Nov 11, 2026
@JacobStephens2
JacobStephens2 marked this pull request as ready for review November 11, 2026 16:53
2026-11-11T17:33:24Z, host dev.etadventures.com. Uncommitted work at the time of failure is included in this commit.
@JacobStephens2

Copy link
Copy Markdown
Owner Author

This was generated by AI during triage.

The Day shift approved recovery of this existing PR through one targeted Fix on Lima, with self-merge disabled, leaving it ready for review. The recovery brief on #1494 preserves the previously agreed behavior and records the interrupted salvage and validation requirements.

Keep #1494 out of Pickup during this recovery: a Fix reserves this PR, not its closing issue. Inspect the salvaged head, reconcile current main and remote branch changes, and validate the final head. Prior green CI on 345a2aff does not validate current head 8bbff4a.

@JacobStephens2 JacobStephens2 added the in-progress A Claim: thirdshift has taken this issue label Nov 11, 2026
@JacobStephens2
JacobStephens2 marked this pull request as ready for review November 11, 2026 17:55
# Conflicts:
#	src/command_limit.rs
#	src/logs.rs
#	src/logs/recording.rs
#	src/main.rs
Quota admission from main probed GitHub (api rate_limit, api user) before
the live-Experiment check, so a Pass made GitHub calls it should skip.
@JacobStephens2 JacobStephens2 removed the in-progress A Claim: thirdshift has taken this issue label Nov 11, 2026
@JacobStephens2
JacobStephens2 merged commit bb84c3a into main Nov 11, 2026
16 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-1494 branch November 11, 2026 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scheduled Passes start Runs during an Experiment on the same host, so Trials time out under load

1 participant