Skip to content

Keep statpopgen's per-sample lists aligned when a sample drops fields - #11404

Merged
robert3005 merged 1 commit into
developfrom
mp/statpopgen-sample-alignment
Nov 12, 2026
Merged

robert3005 merged 1 commit into
developfrom
mp/statpopgen-sample-alignment

Conversation

@mprammer

Copy link
Copy Markdown
Contributor

statpopgen's builder appended each sample's FORMAT fields as noodles yielded them. A sample may drop trailing fields (a no-call is often just ./.), so on rows with no-calls DP, GQ, AD and the other per-sample lists came out shorter than GT, and entry j stopped being sample j. The builder now walks the record's FORMAT keys and takes a dropped field as null.

None of the statpopgen SQL queries read these fields, but the statpopgen compression benchmark from #11043 compresses all of them, so its numbers so far come from the misaligned lists. Cached statpopgen data has to be regenerated to pick up the fix.

🤖 Generated with Claude Code

A sample may drop trailing FORMAT fields (a no-call is often just ./.), and
noodles yields only the fields it has, so on rows with no-calls DP, GQ, AD and
the other per-sample lists came out shorter than GT and entry j stopped being
sample j. Walk the record's FORMAT keys instead, so a dropped field is null.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Martin Prammer <martin@spiraldb.com>
@mprammer mprammer added the changelog/chore A trivial change label Nov 12, 2026
@mprammer
mprammer marked this pull request as ready for review November 12, 2026 06:49
@codspeed

codspeed Bot commented Nov 12, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 27.57%

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 2 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚡ 1 improved benchmark
✅ 1920 untouched benchmarks
⏩ 50 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ CPU Simulation chunked_bool_canonical_into[(1000, 10)] 38.5 µs 30.2 µs +27.57%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing mp/statpopgen-sample-alignment (9f02497) with develop (b6cc108)

Open in CodSpeed

Footnotes

  1. 50 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@robert3005
robert3005 merged commit c3b7be8 into develop Nov 12, 2026
149 checks passed
@robert3005
robert3005 deleted the mp/statpopgen-sample-alignment branch November 12, 2026 08:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants