Repository navigation
Faster version of tscrunch() - #17
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a revised implementation of
Archive.tscrunch()that usesnp.add.reduceat()(see Numpy docs, also StackOverflow) to avoid the slow nestedforloops pointed out in #16.It also gets rid of a bug in the original
tscrunch()implementation where, if the factor didn't divide evenly into the number of subintegrations, the "leftover piece" would just get filled with zeros, never with actual data. For example, if there are 12 subintegrations in the original archive andfactor=5, the final scrunched archive should have 3 subintegrations, where the new subint 0 is the average of old subints 0, 1, 2, 3, and 4, new subint 1 is the average of old subints 5, 6, 7, 8, and 9, and new subint 2 is the average of old subints 10 and 11. In the old version, new subint 2 would be all zeros, but here it actually gets filled in.I have tested against the old implementation on a
.zapfile, and the fractional differences are at the 10^-16 level, consistent with rounding error from doing the additions in a different order -- except, of course, for the "leftover piece". The new implementation took about 172 ms in my testing, as opposed to about 1 minute for the original.