Skip to content

[Firehose Partition] #2286

Closed
Closed

Description

@IceS2

I was wondering if support for Firehose Partition was in the roadmap.
I actually tried to code it myself, but I couldnt build pyarrow locally with pip install -e .

Activity

  1. wesm commented on Jul 19, 2018

    @wesm
    Member

    Are you talking about AWS?

    Installing with pip install -e . is not a supported method. See the documentation about building from source: https://github.com/apache/arrow/blob/master/python/doc/source/development.rst

  2. IceS2 commented on Jul 19, 2018

    @IceS2
    Author

    Hey, yes I'm talking about aws firehose. While hive partitions like year=2018/month=01/day=01 firehose partitions like 2018/01/01

  3. wesm commented on Jul 19, 2018

    @wesm
    Member

    Makes sense. Can you open a JIRA with the feature request?

  4. IceS2 commented on Jul 19, 2018

    @IceS2
    Author

    Hey @wesm , sorry for the delayed answer. I'll gladly open a Jira. Can you just guide me on how to do that? I'm kinda a newb.

    Thanks for your patience.

    I've got another doubt about a feature: ArrowNotImplementedError: lists with structs are not supported.
    Is supporting that kind of nest structures in your plan? any estimate? Thanks!

  5. wesm commented on Jul 19, 2018

    @wesm
    Member

    This is being worked on in apache/parquet-cpp#462

    The Apache JIRA instance is https://issues.apache.org/jira. You'll need to open an account

  6. IceS2 commented on Jul 19, 2018

    @IceS2
    Author

    thanks for the help, created an issue. Not sure it's well made though.
    https://issues.apache.org/jira/browse/ARROW-2882
    If changes need to be made, please let me know

  7. wesm commented on Jul 19, 2018

    @wesm
    Member

    Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions