Repository navigation
FASTQ sorting plugin #53
Description
We previously discussed adding support for sorting FASTQ files by sequence. This would both improve compression (leading to gains in performance and also size on disk) and also potentially improve the preview behaviour particularly for barcoded libraries.
Sorting should only be supported for merged reads (since sorting a set of R1/R2/R3 FASTQ files and leaving them all in the right order seems like too much of a side project).
We will want to make sure that if you re-run a pipeline that already has generated a sorted read file, that it doesn't have to re-sort again since that will just waste a bunch of time.
The plugin should also have an option to sort in reverse order, which mostly just gives the user another try if the "top" of the file is not informative (because it's all the wild type barcodes, for example).
Activity
nickzoic commentedon Mar 13, 2026
This is interesting when considered along with #42 , are there situations where doing a single pass into a more efficient form before trying to do anything else will help us? For example if we scraped up the whole of a FASTQ file and built a sorted / indexed version it would be potentially easier to join later.