[miguel-heygen]

What

When no Studio edit-accuracy shard produced results (for example, the build failed), the gate now says so:

::error::Edit accuracy did not run: 0 of 5 shards reported results; see the shard jobs.

Before, it said Edit accuracy fell against the base branch; see the PR comment., with no PR comment to see, so a build failure read as an accuracy regression. The same goes for a run where some shards did not report and nothing that ran fell (1 of 2 shards reported results). A case that ran and regressed still fails as a drop, even with a shard missing.

Why

ratchet.mjs gate threw no results.json from any shard before writing comment.md or gate.json, so the step exited 1. The workflow's last step printed one fixed message for any non-zero exit. A missing shard also went through the accuracy reasons, so it exited 1 the same way.

Related work

None.

How

• ratchet.mjs owns the verdict. Exit 0 is "holds", exit 1 is "fell" (a case that ran regressed, or the passing count fell), and exit 2 is "did not run" (no shard, or some shards missing while nothing that ran fell).
• It writes comment.md and gate.json in every case, so the sticky PR comment says the gate did not run instead of being skipped. gate.json carries a verdict line.
• The workflow's last step prints that verdict. If the gate crashed before writing it, the step says the gate could not finish and points at its log.

Test plan

• Unit tests added/updated
• Manual testing performed
• Documentation updated (if applicable)
• Comments follow CONTRIBUTING.md "Comments": they say why, not what, and a bug fix says what the code must do and how to reproduce the bug

ratchet.test.mjs, four new tests that run ratchet.mjs gate --shards N as CI does, on temp files:

Case	On main	With this PR
no shard results	fails: no gate.json written (the throw)	exit 2, "did not run: 0 of 5 shards"
1 of 2 shards, nothing fell	fails: exit 1 (read as a drop)	exit 2, "did not run: 1 of 2 shards"
1 of 2 shards, a case that ran regressed	fails: no verdict	exit 1, "fell"
every shard, nothing fell	fails: no verdict	exit 0, "holds"

• Whole file: 21 passed, exit 0, 3 runs in a row.
• Mutations, each turning only the named test red:
  • zero shards exit 1: the no-results test fails;
  • a missing shard counts as a drop: the 1-of-2 holding test fails;
  • a missing shard outranks a real drop: the 1-of-2 regression test fails.
• The workflow's new step was run with a real gate.json (prints its verdict) and with none (prints the "could not finish" line). ci.yml still parses as YAML.

Not exercised: the workflow end to end on GitHub with a failed build. The next CI run with all shards green exercises the "holds" path.

[somanshreddy]

Reviewed at 74fb2c28. No blockers.

Checked

• ratchet.mjs verdict mapping: g.ok false → 1, otherwise a shortfall → 2, otherwise 0. A real drop still outranks a missing shard, and comment.md/gate.json are now written on the zero-shard path too, so the sticky-comment step has a file to post.
• ci.yml: the fail step runs on any non-zero exit_code, so exit 2 still fails the job (fail-closed). jq with 2>/dev/null plus the :- fallback covers a crash before gate.json is written.
• I ran the new gate-command cases plus a probe of my own against the head (scratch copy of ratchet.mjs/report.mjs).

Non-blocking: full grid (merge queue) with a missing shard still reads as "fell"
On the full grid scope is null, so the dead shard's cases count toward basePassing but not toward headPassing. "Passing count fell" then fires, and the code is 1. Probe: base {a, b}, --shards 2, one shard reporting a and holding:

pr   exit 2 | Edit accuracy did not run: 1 of 2 shards reported results; ...
full exit 1 | Edit accuracy fell against the base branch; see the PR comment. | the passing count fell from 2 to 1; 1 of 2 shards reported results

The merge queue runs 20 shards and has no PR comment to point at, so this is where a dead shard is most likely to be misread. A possible fix: when there's a shortfall, drop the passing-count reason (or count missing cases separately) and let only regressed/overclaimed make it a 1. The gate still fails closed, so this is fine as a follow-up.

Nit: VERDICTS[1] says "see the PR comment", but under merge_group there's no PR comment (that was already true before this PR).

— Somu

[github-actions]

Edit accuracy: accurate 271 (base branch 271), smooth 186 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Unstable (1)

• seqplainkeys-none-px-r0-root-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo false, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true