You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This happens pretty reliably when the input_table is a dataset backed by parquet or feather fiels (e.g. input_table returns something like arrow::open_dataset("path/to/{filename}.feather", format = "feather")
One can replicate this by installing an arrowbench branch (voltrondata-labs/arrowbench#37) with, in R: remotes::install_github("ursacomputing/arrowbench@moar-tpch" and then running the following:
Weston Pace / @westonpace:
I should've just looked at your sample as it may have saved me some time. But, I do have a bit more info. The query itself resolves to the following plan (note, we need to do some work on our ToString as it is difficult to make out which node connects to which):
David Li / @lidavidm:
Tangential, but the exec plan string repr being hard to read is unfortunate. It would help if R assigned names to the nodes, or perhaps we could have the ExecPlan auto-assign names if the client bindings don't give any. (Also, maybe GraphViz output would be nice after all…)
Activity
asfimport commentedon Oct 1, 2021
Jonathan Keane / @jonkeane:
I've attached a sample of the R process while it's hung from my machine, in case that's helpful
asfimport commentedon Oct 5, 2021
Weston Pace / @westonpace:
I should've just looked at your sample as it may have saved me some time. But, I do have a bit more info. The query itself resolves to the following plan (note, we need to do some work on our ToString as it is difficult to make out which node connects to which):
The hang is a never-ending busy-loop (i.e. 100% CPU usage) in
arrow::compute::SwissTable::find(this function never returns).I'll attach a GDB stack trace of a sample hang. gdb.log
asfimport commentedon Oct 5, 2021
Weston Pace / @westonpace:
Here is an annotated version of the exec plan.
asfimport commentedon Oct 5, 2021
David Li / @lidavidm:
Tangential, but the exec plan string repr being hard to read is unfortunate. It would help if R assigned names to the nodes, or perhaps we could have the ExecPlan auto-assign names if the client bindings don't give any. (Also, maybe GraphViz output would be nice after all…)
I filed ARROW-14233.
asfimport commentedon Oct 5, 2021
Weston Pace / @westonpace:
I'm attaching a C++ reproduction. It will require some input data. I can host a copy of the input data for a while on Google Drive (only ~200MB): https://drive.google.com/file/d/1kg7Xts6w8Ktzau4Go7euOVSQKdh2qJuG/view?usp=sharing
asfimport commentedon Oct 12, 2021
Neal Richardson / @nealrichardson:
Issue resolved by pull request 11335
#11335