Path.from_uri() doesn't work if the URI contains host component

Bug report

Bug description:

Path.from_uri() introduced in Python 3.13 doesn't work properly if the URI contains a host component other than localhost. Following examples are run with Python 3.13 rc 1 on Linux with a machine having host name kone:

>>> print(Path().from_uri('file:///home/peke/test'))
/home/peke/test
>>> print(Path().from_uri('file://localhost/home/peke/test'))
/home/peke/test
>>> print(Path().from_uri(f'file://{socket.getfqdn()}/home/peke/test'))
//kone/home/peke/test

According to RFC 8089 including the host component as a fully qualified name is fine so this looks like a bug to me.

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

Accepting a host component, other than localhost, raises a question about validity of the used host name. For example, browsers seem to totally ignore the host component and accept paths like file://whatever/home/peke/test in my case. I believe Python should be more strict, though, and raise a ValueError is the host component doesn't match the system where the code is run. Although the RFC mandates the host component to be fully qualified, I believe accepting only the host name should be fine too. If someone wants to parse file URIs with different host names, they can use urilib.parse.urlparse instead.

It might be that UNC Windows file paths even further. I tested this on Windows and there this usage makes sense:

>>> p = Path(r'\\host\path')
>>> print(p.as_uri())
file://host/path/
>>> p == Path.from_uri(p.as_uri())
True

Perhaps from_uri behavior should depend on the operating system.

All URIs with non-empty, non-localhost authorities parse as Path objects that start with a double slash, so it should be straightforward to reject these paths:

path = Path.from_uri('file://server/share/foo.txt')
if path.as_posix().startswith('//'):
    raise ValueError('Non-local file URI')

We could add a local_authorities argument so that users can override the ['', 'localhost'] defaults

Explicitly rejecting paths would certainly be better than returning invalid paths. Including socket.getfqdn() and possibly also socket.gethostname() in the list of local authorities could be more convenient than requiring users to pass them explicitly, though. If the list is made configurable, it probably should contain localhost and the empty string by default.

This probably would anyway require special handling on Windows. On POSIX something like Path.from_uri('file://host/path') should yield Path('/path'), assuming that host is detected to be a local authority, but on Windows throwing the host part away would break UNC paths.

We should try to be consistent across OSs unless we really must diverge IMO.

Explicitly rejecting paths would certainly be better than returning invalid paths.

Paths starting with two slashes are valid on both Windows and POSIX. On Windows they're UNC paths, whereas on POSIX they're implementation-defined (ref).

It can be hard to be totally consistent across OSes. On Windows Path(r'\\host\path').as_uri() yields file://host/path/, and it makes sense that round-trip works and Path.from_uri('file://host/path/') yields Path(r'\\host\path'). In other words, the host component is preserved on Windows. On the other hand, when Path.from_uri('file://host/path/') is used on POSIX, the host component can be validated but there's, AFAIK, no way to preserve it and the return value can only be Path('/path').

There's already other functionality in pathlib that's operating system dependent and I don't see why from_uri couldn't be as well. Someone needing, for example, Windows semantics on POSIX could then explicitly use PureWindowsPath.

I should have used "incorrect" instead of "invalid" in my earlier comment. Although Path('//hello/world') is valid, it certainly isn't the correct return value Path.from_uri('file://hello/world') on POSIX.

pythonGH-123599: Reject non-local authority in `pathlib.Path.from_uri…

Merge branch 'main' into pythongh-123599-reject-non-local

Upon further consideration, I've adjusted my PR so that:

If the authority is empty or 'localhost', we suppress it.
Otherwise, if we're on Windows, we return a UNC path
Otherwise, if the authority resolves to a loopback address, we suppress it.
Otherwise we raise ValueError

This is basically the same logic as urllib.request.FileHandler.file_open(), and so in a future patch I'm planning to slim down the urllib code.

pythonGH-123599: Remove duplicate `url2pathname()` implementation

pythonGH-123599: Deprecate duplicate `pathname2url()` implementation

socket.gethostbyname(socket.gethostname()) raises gaierror(8, 'Name does not resolve').

>>> socket.gethostbyname(socket.gethostname())
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    socket.gethostbyname(socket.gethostname())
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno 8] Name does not resolve

pythonGH-123599: Match `file:` URL hostname against machine hostname …

GH-123599: Match `file:` URL hostname against machine hostname in url…

The above test failure was fixed in #132523

Re-opening to discuss this point from Serhiy on #126844:

url2pathname() now performs network requests (and hang for a time).

Having given it more thought, I reckon we should call gethostbyname() only when a new resolve_netloc keyword-only argument is set to true (default false). Existing users of this function probably don't expect it to perform network access, and it could have larger consequences in environments where DNS resolution is borked.

Thoughts / 👍 / 👎?

You just read my mind. I was thinking about suggesting adding a keyword argument for resolving hostname, but then I thought that url2pathname() may be not much used in third-party code, it may be not worth a hassle.

But adding an option is safer. In future we can even change its default value, but the users will be able to request the old behavior.

pythonGH-123599: `url2pathname()`: don't call `gethostbyname()` by de…

Also FYI, I'm seeing idna encoding errors on one of my private test machines from the address = socket.gethostbyname(authority) line. I'm not entirely sure what path is being passed in to cause it, but I suspect we ought to swallow UnicodeEncodeError in _is_local_authority anyway?

Merge branch 'main' into pythongh-123599-resolve-netloc

GH-123599: `url2pathname()`: don't call `gethostbyname()` by default (#…

pythonGH-123599: `url2pathname()`: don't call `gethostbyname()` by de…

Navigation Menu

Repository navigation

`Path.from_uri()` doesn't work if the URI contains host component #123599

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

11 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Path.from_uri() doesn't work if the URI contains host component #123599

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

pekkaklarck commented on Sep 2, 2024

barneygale commented on Sep 2, 2024

pekkaklarck commented on Sep 2, 2024

pekkaklarck commented on Sep 2, 2024

barneygale commented on Sep 2, 2024

pekkaklarck commented on Sep 2, 2024

barneygale commented on Oct 21, 2024

11 remaining items

serhiy-storchaka commented on Apr 14, 2025

barneygale commented on Apr 15, 2025

barneygale commented on Apr 16, 2025

serhiy-storchaka commented on Apr 16, 2025

zooba commented on Apr 16, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

`Path.from_uri()` doesn't work if the URI contains host component #123599