Skip to content

[C++] arrow-compute-hash-join-node-test failed #30435

Closed

Description

@asfimport

arrow-compute-hash-join-node-test failed in my environment (windows, msvc 19.12.25835)

because in HashJoinBasicImpl::ProbeBatch_Lookup hash_table_ is empty, but hash_table_empty_ is false. 

I workaround this issue using the attached patch, but fix likely should be different.

Reporter: Renat Valiullin / @rip-nsk
Assignee: Michal Nowakiewicz / @michalursa

Related issues:

Original Issue Attachments:

PRs and other links:

Note: This issue was originally created as ARROW-14911. Please see the migration documentation for further details.

Activity

  1. asfimport commented on Nov 30, 2021

    @asfimport
    Author
  2. asfimport commented on Nov 30, 2021

    @asfimport
    Author

    Sasha Krassovsky / @save-buffer:
    Hi [~rip.nsk@gmail.com]

    Could you please share the parameters you gave to the join and what data you used?

    Thanks!

  3. asfimport commented on Dec 1, 2021

    @asfimport
    Author

    Renat Valiullin / @rip-nsk:
    @save-buffer,

    I don't know how to get this data, is there way to save shuffled_input_arrays into file?

    TEST(HashJoin, Random) prints

    RIGHT_SEMI IS_IS int[0]int[0]
    payload_l: str[7..36]int[1]str[13..38]
    payload_r: int[1]str[12..15]
    num_rows_l = 17242 num_rows_r = 16832 batch size = 75 parallel = true
    before HashJoinWithExecPlan fails.

    BTW, workaround patch does not help.

  4. asfimport commented on Dec 7, 2021

    @asfimport
    Author

    Antoine Pitrou / @pitrou:
    @save-buffer @michalursa Can one of you perhaps create a draft branch or PR with some additional logging to help diagnosing what happens on [~rip.nsk@gmail.com]'s setup?

    Also, it seems the code is almost uncommented and undocumented in hash_join.(h|cc), it would be nice if you could create a PR to improve readability and maintainability there.

  5. asfimport commented on Dec 7, 2021

    @asfimport
    Author

    Sasha Krassovsky / @save-buffer:
    One thing is that this code will be gone in a few weeks (Michal is rewriting hash join completely). For now I'll try to reproduce it.

  6. asfimport commented on Jan 13, 2022

    @asfimport
    Author

    Weston Pace / @westonpace:
    I'm not sure the input data will be as important. If I am reading the test correctly it initializes a random generator with a fixed seed. So even though the test is testing random data it should be deterministic between runs. I have verified that we are running this test on our CI servers (which includes VS2017 although at a higher version than 19.12).

    The test does do some parallel execution so there is a possibility it is a race condition causing the flakiness.

    I also built locally with VS2019 in both debug mode and release and tried running this test both with and without CPU stress and have not been able to reproduce it. I was doing this against the latest master and not the 6.0.1 tag so that might be a factor. I will try and reproduce this some more tomorrow with VS2017.

    [~rip.nsk@gmail.com] How reliable is this failure? Does it fail every time or intermittently? When it fails does it always display the exact same output? (e.g. RIGHT SEMI IS_IS...)

    At the moment I'm not sure this qualifies as a blocker.

  7. asfimport commented on Jan 14, 2022

    @asfimport
    Author

    Weston Pace / @westonpace:
    I ran tests against the 6.0.1 tag without any failures. I then downgraded to VS 2017 but I was not able to get the exact same version. Microsoft only makes versions 19.10 and 19.15 available for download. 19.10 fails to compile and I am unable to reproduce the failure with 19.15.

    If the failure is reliable it's possible that there is a hardware mismatch. The join node makes heavy use of intrinsics and although they are guarded by ARROW_HAVE_AVX2 there might be some odd corner case.

    [~rip.nsk@gmail.com] Do you know what kind of CPU you are running on?

    I'm going to move this from blocker to major.

  8. asfimport commented on Jan 17, 2022

    @asfimport
    Author

    Krisztian Szucs / @kszucs:
    Postponing it to 8.0 then.

  9. asfimport commented on Apr 21, 2022

    @asfimport
    Author

    Weston Pace / @westonpace:
    Issue resolved by pull request 12894
    #12894

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions