[tannevaled]

The tool compared passing counts. A file that goes from no examples to
many failing examples has a pass delta of zero and never appeared in the
report — not even without -quiet.

That is not hypothetical: it cost a published figure. Measuring the 4.0 version
move, these were invisible:

core/binding/implicit_parameter_get_spec.rb        0 → 0   +0   fail+error 0 → 14
core/binding/implicit_parameter_defined_spec.rb    0 → 0   +0   fail+error 0 → 13
core/binding/implicit_parameters_spec.rb           0 → 0   +0   fail+error 0 → 10
core/marshal/dump_spec.rb                      191 → 191   +0   fail+error 17 → 20
language/predefined_spec.rb                    141 → 141   +0   fail+error 20 → 24

The three core/binding/ files are pass=0 fail=0 error=0 at 3.4.1 — every
example behind a ruby_version_is "4.0" guard — and 37 failures at 4.0.9, where
the guard opens and three new Binding methods turn out not to exist. "Ten
examples of work" was short by about sixty-five, and #799 is corrected.

Six more files were reported as gains while also gaining failures. The worst
reads +3 in the old output:

core/thread/raise_spec.rb   48 → 51   +3   ⛔ fail+error 10 → 26

Nineteen examples arrive at 4.0; three pass and sixteen fail. A pass-only delta
cannot say that, and it was quoted as though it could.

The change

A row prints whenever either the passes moved or the failures did, carrying
⛔ fail+error A → B, and the summary counts the pass-neutral ones separately:
N gained failures without losing passes. They are named apart because such a
file is not a regression in the passing total and would otherwise read as
nothing at all.

The test asserts that -quiet does not suppress them, since a delta of zero
is exactly how they hid, and keeps the control beside it: a file quiet on both
engines must still stay out of a -quiet report, or every unchanged file would
print.

⛔ The lesson behind it was already written down and not applied: ask what the
target version adds before measuring what moves. A delta cannot see an API
absent on both sides of the comparison when the specs for it are guarded away on
one of them.

Refs #799

🤖 Generated with Claude Code