Skip to content

ARROW-17278: [C++][Benchmarking] Add AsofJoin Ordering Assertion and Benchmark Fixes - #13771

Closed
iChauster wants to merge 3 commits into
apache:mainfrom
iChauster:asof_join_thread_per_source
Closed

iChauster wants to merge 3 commits into
apache:mainfrom
iChauster:asof_join_thread_per_source

Conversation

@iChauster

@iChauster iChauster commented Aug 1, 2022 •

Copy link
Copy Markdown
Contributor
  • Add a invalid status if batches are ingested in an incorrect order for any source in AsOfJoin.
  • Fix benchmarks to use isolated memory pools.
  • Create and discard memory pools over different benchmark iterations (if possible) to prevent OOM from creating too many memory pools.

@github-actions

github-actions Bot commented Aug 1, 2022

Copy link
Copy Markdown

@github-actions

github-actions Bot commented Aug 1, 2022

Copy link
Copy Markdown

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

@iChauster

Copy link
Copy Markdown
Contributor Author

@westonpace Hi Weston,

I wrote some new tests to validate that our in-order assertions are working (it returns an invalid status which is caught by ErrorIfNotOk) . They seem to detect the out-of-ordered test locally on my computer and it passes, but it seems to fail on the CI checks -- any idea why?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We raise the Invalid case here, and the status is processed by ErrorIfNotOk in InputReceived

@iChauster

iChauster commented Aug 2, 2022 •

Copy link
Copy Markdown
Contributor Author

I think I found the issue, doesn't seem like it passes always?

EDIT: seems like some sort of race condition. We only run the check when are sure the queue has batches in it, however, it seems like in the cases in which the tests fails, it is always empty. The execution context is setup with nullptr as well. Any ideas here @icexelloss ?

@icexelloss

icexelloss commented Aug 2, 2022 via email

Copy link
Copy Markdown
Contributor

@iChauster

iChauster commented Aug 3, 2022 •

Copy link
Copy Markdown
Contributor Author

Resolved, was checking the last batch in the queue to find latest time, however these batches get consumed and the queue can become empty, instead introduce another instance variable to track latest time seen in the InputState.

@westonpace westonpace left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a nice check to add. Thanks. CI has a way of running parallel tasks with pretty much the oddest scheduling you could possibly imagine so that is generally my first guess.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't remember if this kind of access outside the mutex causes TSAN errors or not. Have you run TSAN locally?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey Weston, I'm not sure how to run TSAN locally with ninja, is there some documentation on how to do this?

In any case, I don't actually think we use this method anywhere -- I'm going to remove this for now.

@iChauster
iChauster requested a review from westonpace August 4, 2022 17:40

@westonpace westonpace left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few minor style / nitpick requests.

@iChauster
iChauster force-pushed the asof_join_thread_per_source branch from 36f5917 to 0c3707f Compare August 5, 2022 14:54

@icexelloss icexelloss left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iChauster Looks good I have a few minor comments please take a look

@icexelloss

Copy link
Copy Markdown
Contributor

@westonpace - @iChauster has finished his internship last week so I will take this PR over. Let me know if you have more comments.

@icexelloss

Copy link
Copy Markdown
Contributor

@westonpace Are there more changes needed for this PR?

@icexelloss

Copy link
Copy Markdown
Contributor

@westonpace Anything else we need to do here for merging?

@amol-

amol- commented Mar 30, 2023

Copy link
Copy Markdown
Member

Closing because it has been untouched for a while, in case it's still relevant feel free to reopen and move it forward 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants