Repository navigation
OSError when try convert URI to pathlib.Path #91504
Description
Activity
Please show what the output in 3.10 is.
Python 3.10 output:
E:\Work\Venvs\p310\Scripts\python.exe "E:/uri_test/pathlib_test.py" D:\Tests\samples\images\3.bmp exists=True file:///D:/Tests/samples/images/3.bmp D:Tests\samples\images\3.bmp exists=True Process finished with exit code 0
Now I see that one '\' is missing in path after drive letter:
D:Tests\samples\images\3.bmp exists=TrueA
pathlib.Pathcannot be a URI or be constructed from a URI. For example:>>> p = pathlib.Path('file:///C:/Temp/test.txt') >>> p.drive '' >>> p.root '' >>> os.fspath(p) 'file:\\C:\\Temp\\test.txt
The path was parsed as an invalid relative path. Perhaps there's a case for adding a
from_uri()constructor.In 3.10+,
resolve()'succeeds' with an invalid path such as this because it defaults to callingrealpath()in non-strict mode, which in this case is garbage-in garbage-out. For example:>>> p.resolve() WindowsPath('C:Temp/test.txt') >>> os.path.realpath(p) 'C:Temp\\test.txt'The above result is a drive-relative path, i.e. it has no root path and thus depends on the working directory on drive "C:".
I think this method
from_uri()is a good idea.>>> p = pathlib.Path().from_uri("file:///D:/Tests/samples/images/3.bmp") >>> p.resolve() 'D:\Tests\samples\images\3.bmp'
There is a problem with URI -> pathlib.Path -> URI conversion:
>>>p = Path(r"file:///D:/Tests/samples/images/3.bmp").resolve() >>>p WindowsPath('D:Tests/samples/images/3.bmp') >>>p.as_uri() Traceback (most recent call last): File "C:\pythons\Python310\lib\code.py", line 90, in runcode exec(code, self.locals) File "<input>", line 1, in <module> File "C:\pythons\Python310\lib\pathlib.py", line 649, in as_uri raise ValueError("relative path can't be expressed as a file URI") ValueError: relative path can't be expressed as a file URI
As @eryksun said: "A
pathlib.Pathcannot be a URI or be constructed from a URI".A solution that works today:
from urllib.request import url2pathname from pathlib import Path def path_from_uri(uri): return Path(url2pathname(uri.removeprefix('file:')))
I'd support the addition of a
Path.from_uri()classmethod, on the condition that we makefrom_uri()andas_uri()call through tourl2pathnameandpathname2urlinurllib.request, rather than adding another duplicate implementation.Reacted by Simão Afonso @ Powertools TechI suspect that the rules for parsing/emitting URLs will vary depending on whether the path uses Windows or POSIX semantics. If that's the case, I'd like to propose the following:
- A new
os.path.fileuri()function, implemented inposixpathandntpath.nturl2path.pathname2uri()callsntpath.fileuri()urllib.request.pathname2uri()callsos.path.fileuri()pathlib.PurePosixPath.as_uri()callsposixpath.fileuri()pathlib.PureWindowsPath.as_uri()callsntpath.fileuri()
- A new
os.path.parsefileuri()function (unsure of naming). Similarly:nturl2path.uri2pathname()callsntpath.parsefileuri()urllib.request.uri2pathname()callsos.path.parsefileuri()pathlib.PurePosixPath.from_uri()callsposixpath.parsefileuri()pathlib.PureWindowsPath.from_uri()callsntpath.parsefileuri().
From a pathlib perspective this makes sense to me, as I'm hoping to move towards wrapping
os.pathmore directly - see #31691.Thoughts?
- A new
I'm closing this issue because the
pathlib.Pathconstructor isn't intended to support file URIs.@barneygale, I think the suggested changes to
os.pathshould be discussed in a forum, even if just to bike shed the function names.Reacted by Barney GaleIf this is true, then the Path constructor should raise an exception when it's given a file URI, so that it doesn't give the appearance of supporting file URIs when in fact it doesn't.
It's a good idea, but file URIs are also valid POSIX paths! You can
mkdir -p file:///foo/bar/bazfrom your terminal today.Seems relatively unlikely someone would actually want that though?
As a user, I want to be able to throw any string at
pathlib.Pathand have it efficiently parse it as a path. It doesn't currently perform any validation beyond checking forbytes. If we attempt to detect and rejectfile:URIs, it will slow things down and open the door to validating other things, which will further complicate and slow pathlib.Reacted by Alex WaygoodThe current implementation is, IMO, the worst option: leave
file:as the first element in the path. That is utterly useless (is_file()will always returnFalse). As I see it, there are two relatively simple options that would make it at least somewhat useful:- Remove the
file:part and just create a path based on the rest. This would be the most useful since it would generate a potentially valid/useful path. Thinking about it, this is probably my personal preference and further supports the "throw any string" at Path instances and have it parse usable paths. - Raise an exception - since a Path instance that includes
file:as the first element is utterly useless, this would at least notify users that it's not going to generate a valid Path.
- Remove the
That is utterly useless (
is_file()will always returnFalse)It works fine for me on Linux:
$ touch 'file:' $ ./python >>> import pathlib >>> pathlib.Path('file:').is_file() TrueI agree with @barneygale : There's no reason to prohibit filenames starting with "file:". They're perfectly valid, at least on POSIX systems.
Also, I don't think we'd want to implement any "strip of leading 'file:' characters" logic just on Windows.
Raise an exception - since a Path instance that includes file: as the first element is utterly useless
To clarify the case on Windows, the NTFS and ReFS filesystems support file streams of the form "filename:streamname:streamtype", "filename:streamname" ("$DATA" stream type), and "filename::streamtype" (anonymous or default stream name for the given stream type). But just "file:" without a stream name or stream type is invalid, and "file:///foo/bar/baz" is invalid because "///foo/bar/baz" is parsed as the stream name, and stream names reserve "/", ":", and NUL as invalid name characters.
When try to convert URI to pathlib.Path I get an error:
Code to reproduce:
Python version: 3.9
With 3.10 works.