You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I was wondering if support for Firehose Partition was in the roadmap.
I actually tried to code it myself, but I couldnt build pyarrow locally with pip install -e .
Hey @wesm , sorry for the delayed answer. I'll gladly open a Jira. Can you just guide me on how to do that? I'm kinda a newb.
Thanks for your patience.
I've got another doubt about a feature: ArrowNotImplementedError: lists with structs are not supported.
Is supporting that kind of nest structures in your plan? any estimate? Thanks!
Activity
wesm commentedon Jul 19, 2018
Are you talking about AWS?
Installing with
pip install -e .is not a supported method. See the documentation about building from source: https://github.com/apache/arrow/blob/master/python/doc/source/development.rstIceS2 commentedon Jul 19, 2018
Hey, yes I'm talking about aws firehose. While hive partitions like year=2018/month=01/day=01 firehose partitions like 2018/01/01
wesm commentedon Jul 19, 2018
Makes sense. Can you open a JIRA with the feature request?
IceS2 commentedon Jul 19, 2018
Hey @wesm , sorry for the delayed answer. I'll gladly open a Jira. Can you just guide me on how to do that? I'm kinda a newb.
Thanks for your patience.
I've got another doubt about a feature:
ArrowNotImplementedError: lists with structs are not supported.Is supporting that kind of nest structures in your plan? any estimate? Thanks!
wesm commentedon Jul 19, 2018
This is being worked on in apache/parquet-cpp#462
The Apache JIRA instance is https://issues.apache.org/jira. You'll need to open an account
IceS2 commentedon Jul 19, 2018
thanks for the help, created an issue. Not sure it's well made though.
https://issues.apache.org/jira/browse/ARROW-2882
If changes need to be made, please let me know
wesm commentedon Jul 19, 2018
Thank you