Skip to content

Faster version of tscrunch() - #17

Merged
mtlam merged 1 commit into
mtlam:masterfrom
rossjjennings:improved-tscrunch
Jun 11, 2024
Merged

mtlam merged 1 commit into
mtlam:masterfrom
rossjjennings:improved-tscrunch

Conversation

@rossjjennings

Copy link
Copy Markdown
Contributor

This is a revised implementation of Archive.tscrunch() that uses np.add.reduceat() (see Numpy docs, also StackOverflow) to avoid the slow nested for loops pointed out in #16.

It also gets rid of a bug in the original tscrunch() implementation where, if the factor didn't divide evenly into the number of subintegrations, the "leftover piece" would just get filled with zeros, never with actual data. For example, if there are 12 subintegrations in the original archive and factor=5, the final scrunched archive should have 3 subintegrations, where the new subint 0 is the average of old subints 0, 1, 2, 3, and 4, new subint 1 is the average of old subints 5, 6, 7, 8, and 9, and new subint 2 is the average of old subints 10 and 11. In the old version, new subint 2 would be all zeros, but here it actually gets filled in.

I have tested against the old implementation on a .zap file, and the fractional differences are at the 10^-16 level, consistent with rounding error from doing the additions in a different order -- except, of course, for the "leftover piece". The new implementation took about 172 ms in my testing, as opposed to about 1 minute for the original.

@mtlam
mtlam merged commit e44ce97 into mtlam:master Jun 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants