Skip to content

Fix QuerySet slicing logic - #1047

Merged
jacalata merged 3 commits into
tableau:developmentfrom
jorwoods:jorwoods/queryset_slicing
Jun 2, 2022
Merged

jacalata merged 3 commits into
tableau:developmentfrom
jorwoods:jorwoods/queryset_slicing

Conversation

@jorwoods

Copy link
Copy Markdown
Contributor

The old logic for handling slices on QuerySet objects worked for toy examples when the result set was small, but would crash if having to work through too many items. This logic will work for all sizes of results.

@jorwoods jorwoods mentioned this pull request Jun 1, 2022
@jacalata
jacalata force-pushed the development branch 2 times, most recently from fd17ecc to ec70791 Compare June 1, 2022 20:28
@jacalata

jacalata commented Jun 2, 2022

Copy link
Copy Markdown
Contributor

looks good, I know I've made it a little ugly with the rebasing.

Is it possible to write a test that shows this fix? What kind of sizes were you seeing the crash at?

@jacalata

jacalata commented Jun 2, 2022

Copy link
Copy Markdown
Contributor

(I will fix the conflicts/merge it, don't worry about those)

@jorwoods
jorwoods force-pushed the jorwoods/queryset_slicing branch from edaac6b to 60eeb37 Compare June 2, 2022 12:48
@jorwoods
jorwoods force-pushed the jorwoods/queryset_slicing branch from 8fdbbdb to 9e476d5 Compare June 2, 2022 13:50
@jorwoods

jorwoods commented Jun 2, 2022

Copy link
Copy Markdown
Contributor Author

The real issue was requesting an index that was a multiple of the page size, which would happen when trying to span pages when slicing. It would get stuck in an infinite loop whenever the requested item number is a multiple of the page size. Fixed this by incrementing the k number by 1 whenever the requested index is not in the current cache's range. This makes it so that the function will increment to the next page as appropriate when the slice requires it.

@jacalata

jacalata commented Jun 2, 2022

Copy link
Copy Markdown
Contributor

Huh, this reminds me of a recently discovered server bug where attempting to iterate over more than 10,000 datasources was giving an error. Does that sound like it could be related to this one having issues, or just coincidence?

@jacalata
jacalata merged commit 2dce2ef into tableau:development Jun 2, 2022
@jorwoods

jorwoods commented Jun 2, 2022

Copy link
Copy Markdown
Contributor Author

@jacalata Unrelated. This was me making a simple logic/math mistake when I initially wrote the slicing logic, and I did not create a proper test to span pages and catch it.

@jorwoods
jorwoods deleted the jorwoods/queryset_slicing branch June 7, 2024 11:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants