Skip to content

Close fastlanes test gaps found by mutation testing - #11475

Merged
robert3005 merged 4 commits into
developfrom
ji/fastlanes-mutation-tests
Nov 15, 2026
Merged

robert3005 merged 4 commits into
developfrom
ji/fastlanes-mutation-tests

Conversation

@joseph-isaacs

@joseph-isaacs joseph-isaacs commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Mutation testing with cargo-mutants (see #11471) found code in vortex-fastlanes that the tests run but never check. This PR adds tests for two such gaps, keeping only the cases that catch distinct mutants. It changes tests only.

Changes

Bit-packed is_constant patch path

No test reached the kernel's patch handling. The only test, is_constant_with_patches, called the public is_constant, which answered from min/max statistics before dispatching to the kernel. Deleting apply_patches, or changing any of its bounds, went unnoticed: 22 of the 26 viable mutants in the file survived. A temporary panic! on the patch line confirmed that no test executed it.

The test is replaced by kernel_applies_patches, which calls BitPackedIsConstantKernel directly, following the delta kernel's tests. It has three rstest cases at bit width 2:

  • all_patched: 3000 patched values, filling two full chunks and a trailer. Only this case catches the comparison of later chunks against the first chunk's value.
  • patched_inside_sliced: the slice 100..2900, whose partial first chunk, full chunk and trailer all hold patches, with packed values outside the slice.
  • patch_in_one_row_trailer: 1025 rows, with the only patch in a one-row trailer. Only this case catches a mutated trailer range (current_idx * chunk_len).

I chose these three from 12 candidates using a kill matrix built from cargo-mutants' per-mutant logs: they are a minimum set that catches every viable mutant. Each case states its expected answer and also checks is_constant on the array against is_constant on the source data. The cases slice through <BitPacked as SliceKernel>::slice, because ArrayRef::slice stays a lazy SliceArray over patched arrays.

Result for bitpacking/compute/is_constant.rs: 22 missed and 4 caught before; 0 missed and 26 caught after.

Delta take of more than a chunk from one chunk

Repeated indices can select more positions from one chunk than the chunk holds. take_grouped groups positions by chunk without capping a group, and only its whole-chunk decode can handle such a group: forcing the selective decode overruns its 1024-value buffer. No delta take test selected that many positions from one chunk, so the mutants that force selective decoding (> to < and to ==) survived.

take_more_positions_than_a_chunk_from_one_chunk takes 1100 repeated positions from one chunk of a 9000-row array, which is few enough to stay on the grouped path, and compares the result with the decoded take.

Result for take_chunks.rs and delta/compute/take.rs: 7 survivors before.

  • The 2 take_grouped survivors are now caught.
  • 1 is equivalent: the Mask::AllTrue arm is a fast path with an equal result. Add cargo-mutants mutation testing: CI workflow and agent skill #11471 records it as an exclusion.
  • 4 are deferred, all in the Delta::take full-decode heuristic. Both paths give equal results, but they handle an out-of-bounds index differently: the grouped path errors at take, while the full-decode path returns a lazy take that errors only when executed. A test that kills these mutants would pin that timing rather than a result.

Checks run:

  • cargo clippy -p vortex-fastlanes --all-targets --all-features -- -D warnings
  • cargo +nightly-2026-11-13 fmt -p vortex-fastlanes --check
  • cargo-mutants on both scopes before and after, with the crate's full test suite. On develop, cargo-mutants needs a temporary parse workaround for unpack_iter.rs, which Add cargo-mutants mutation testing: CI workflow and agent skill #11471 makes permanent.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UeniBCBxF4oRz7efstrfC9

Mutation testing showed that no test reached this kernel's patch handling:
the only test called `is_constant`, which answered from min/max stats before
dispatching, so deleting `apply_patches` or changing any of its bounds went
unnoticed (22 of 26 viable mutants in the file survived).

Call the kernel directly, as the delta kernel's tests do, with patches in a
sliced header, full chunks, a trailer, a one-row trailer and outside a slice,
and check each answer against `is_constant` on the source. All 26 viable
mutants in the file are now caught.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBCBxF4oRz7efstrfC9
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs added the changelog/chore A trivial change label Nov 13, 2026 — with Claude
@codspeed

codspeed Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 1 benchmark is dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

✅ 1936 untouched benchmarks
⏩ 50 skipped benchmarks1


Comparing ji/fastlanes-mutation-tests (af5f308) with develop (d7aa24e)

Open in CodSpeed

Footnotes

  1. 50 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

Mutation testing showed that no delta take selected more positions from one
chunk than the chunk holds, which repeated indices allow. `take_grouped`
groups positions by chunk without capping a group, and only its whole-chunk
decode handles such a group: forcing the selective decode overruns its
1024-value buffer, yet those mutants (`>` to `<` and `==`) survived.

Take 1100 repeated positions from one chunk of a 9000-row array, few enough
to stay on the grouped path, and compare with the decoded take.

Mutation result for take_chunks.rs and delta/compute/take.rs: the 2
take_grouped survivors are caught. Of the other 5, the `AllTrue` arm is a
fast path with an equal result, and the 4 in the full-decode heuristic
differ only in whether an out-of-bounds index errors at take or when the
lazy take executes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBCBxF4oRz7efstrfC9
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs changed the title Test the bit-packed is_constant kernel's patch path Close fastlanes test gaps found by mutation testing Nov 13, 2026
joseph-isaacs and others added 2 commits November 13, 2026 23:37
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBCBxF4oRz7efstrfC9
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
A kill matrix from cargo-mutants' per-mutant logs showed that 3 of the 12
cases catch all 26 viable mutants of the kernel: an all-patched array
(the only case that catches the cross-chunk comparison), a slice whose
header, chunks and trailer hold patches, and a one-row trailer (the only
case that catches the trailer range). Drop the other 9.

Mutation result with the full crate suite: 26 caught, 0 missed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBCBxF4oRz7efstrfC9
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@robert3005
robert3005 merged commit e2871c2 into develop Nov 15, 2026
80 checks passed
@robert3005
robert3005 deleted the ji/fastlanes-mutation-tests branch November 15, 2026 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants