Skip to content

File URI Scheme Interpretation #2562

Closed

Description

@tustvold

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

The file URI scheme implemented in ObjectStoreRegistry does not appear to follow the specification described in RFC 3986.

In particular:

  • It does not handle the host component
  • It accepts non-absolute paths

Describe the solution you'd like

The file:// URIs should be handled in a spec-compliant way. Unfortunately this on its own would prevent the use of local paths, which would likely be annoying for users.

I would therefore propose we special case URIs without a scheme, and canonicalise them within ObjectStoreRegistry.

Describe alternatives you've considered

We could not do this

Additional context

I encountered this whilst working on #2489, as the object_store crate purposefully does not handle relative paths.

Thoughts @yahoNanJing @thinkharderdev

Activity

  1. thinkharderdev commented on May 17, 2022

    @thinkharderdev
    Contributor

    Makes sense. This also came up in apache/datafusion-ballista#481 where we had a regression when we stripped the schema from file URIs. We actually use the scheme to resolve the correct ObjectStore when deserializing plans in Ballista.

  2. alamb commented on May 17, 2022

    @alamb
    Contributor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions