Skip to content

[Windows] OSError when testing whether pathlib.Path('*') exists #79487

Description

@jimbo1qaz
mannequin
BPO 35306
Nosy @pfmoore, @pitrou, @tjguk, @zware, @serhiy-storchaka, @eryksun, @zooba, @miss-islington
PRs
  • bpo-35306: Handle '*' in pathlib.Path functions on Windows #11133
  • bpo-35306: Avoid raising OSError from pathlib.Path.exists when passed an invalid filename #25529
  • [3.9] bpo-35306: Avoid raising OSError from pathlib.Path.exists when passed an invalid filename (GH-25529) #25534
  • [3.8] bpo-35306: Avoid raising OSError from pathlib.Path.exists when passed an invalid filename (GH-25529) #25535
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/zooba'
    closed_at = <Date 2021-04-22.20:12:38.885>
    created_at = <Date 2018-11-24.05:55:59.185>
    labels = ['type-bug', '3.8', '3.9', '3.10', 'library', 'OS-windows']
    title = "[Windows] OSError when testing whether pathlib.Path('*') exists"
    updated_at = <Date 2021-04-22.20:53:11.398>
    user = 'https://bugs.python.org/jimbo1qaz'

    bugs.python.org fields:

    activity = <Date 2021-04-22.20:53:11.398>
    actor = 'steve.dower'
    assignee = 'steve.dower'
    closed = True
    closed_date = <Date 2021-04-22.20:12:38.885>
    closer = 'steve.dower'
    components = ['Library (Lib)', 'Windows']
    creation = <Date 2018-11-24.05:55:59.185>
    creator = 'jimbo1qaz_'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 35306
    keywords = ['patch']
    message_count = 8.0
    messages = ['330371', '330394', '333092', '333728', '333730', '391627', '391630', '391631']
    nosy_count = 9.0
    nosy_names = ['paul.moore', 'pitrou', 'tim.golden', 'zach.ware', 'serhiy.storchaka', 'eryksun', 'steve.dower', 'miss-islington', 'jimbo1qaz_']
    pr_nums = ['11133', '25529', '25534', '25535']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue35306'
    versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

    Activity

    1. jimbo1qaz commented on Nov 24, 2018

      jimbo1qazmannequin
      MannequinAuthor

      I'm writing a program taking paths from user input through CLI.

      path is a pathlib.Path().

      Since Windows doesn't expand asterisks, I check if the path doesn't exist. If so I expand using Path().glob(path).

      Unfortunately on Windows, if path (type: Path) contains asterisks, checking path.exists() or path.is_dir() raises WinError 123.

      Python 3.7.0 (default, Jun 28 2018, 08:04:48) [MSC v.1912 64 bit (AMD64)] :: Anaconda, Inc. on win32
      Type "help", "copyright", "credits" or "license" for more information.
      >>> from pathlib import Path
      >>> Path('*').exists()
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
        File "C:\Users\jimbo1qaz\Miniconda3\envs\python37\lib\pathlib.py", line 1318, in exists
          self.stat()
        File "C:\Users\jimbo1qaz\Miniconda3\envs\python37\lib\pathlib.py", line 1140, in stat
          return self._accessor.stat(self)
      OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '*'
      >>> Path('*').is_dir()
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
        File "C:\Users\jimbo1qaz\Miniconda3\envs\python37\lib\pathlib.py", line 1330, in is_dir
          return S_ISDIR(self.stat().st_mode)
        File "C:\Users\jimbo1qaz\Miniconda3\envs\python37\lib\pathlib.py", line 1140, in stat
          return self._accessor.stat(self)
      OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '*'

      I also reproduced on Miniconda 3.6.6, 3.7.0, and official Python 3.7.1.

      According to https://bugs.python.org/issue29827 , os.path.exists() (not Path.exists() ) returns False on any OSError.

      -----------------

      On Linux, checking paths with null bytes (a less common occurrence) raises a different error:

      >>> import pathlib
      >>> pathlib.Path("\x00").exists()
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
        File "/usr/lib/python3.6/pathlib.py", line 1336, in exists
          self.stat()
        File "/usr/lib/python3.6/pathlib.py", line 1158, in stat
          return self._accessor.stat(self)
        File "/usr/lib/python3.6/pathlib.py", line 387, in wrapped
          return strfunc(str(pathobj), *args)
      ValueError: embedded null byte
    2. added
      stdlibStandard Library Python modules in the Lib/ directory
      on Nov 24, 2018
    3. eryksun commented on Nov 25, 2018

      @eryksun
      Contributor

      Path.exists should ignore all OSError exceptions, as os.path.exists does. Barring that, I suppose for Windows we could add EINVAL to IGNORED_ERROS.

      The problem with ValueError was already addressed in bpo-33721.

    4. jimbo1qaz commented on Jan 6, 2019

      jimbo1qazmannequin
      MannequinAuthor

      Should Path.resolve() also avoid raising OSError?

      Path('*').resolve()

      Traceback (most recent call last):
      ...truncated
        File "<ipython-input-5-4fa2fec5c8b3>", line 1, in <module>
          Path('*').resolve()
        File "C:\Users\jimbo1qaz\AppData\Local\Programs\Python\Python37\lib\pathlib.py", line 1134, in resolve
          s = self._flavour.resolve(self, strict=strict)
        File "C:\Users\jimbo1qaz\AppData\Local\Programs\Python\Python37\lib\pathlib.py", line 192, in resolve
          s = self._ext_to_normal(_getfinalpathname(s))
      OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '*'
      os.path.realpath('"*')
      Out[8]: 'C:\\Users\\jimbo1qaz\\Dropbox\\encrypted\\code\\corrscope\\"*'
      os.path.abspath('*"')
      Out[13]: 'C:\\Users\\jimbo1qaz\\Dropbox\\encrypted\\code\\corrscope\\*"'

      (sidenote: what os.path operation does Path.resolve() match? Path('nonexistent').resolve() returns a relative path on Python 3.7.1, whereas Path().resolve() returns an absolute path.)

    5. zooba commented on Jan 15, 2019

      @zooba
      Member

      Pathlib doesn't necessarily directly follow os on its error handling - adding Antoine for comment.

      Passing strict=False to resolve() should be able to handle an invalid name like that. If not, I propose that we change it so that it does.

    6. pitrou commented on Jan 15, 2019

      @pitrou
      Member

      I'm fine with swallowing the error in both exists() and resolve(). We should be careful not to swallow errors too broadly, though. The code paths should be audited to check that EINVAL can't mean something else.

    7. added and removed on Mar 27, 2021
    8. changed the title [-]OSError [WinError 123] when testing if pathlib.Path('*') (asterisks) exists[/-] [+][Windows] OSError when testing whether pathlib.Path('*') exists[/+] on Mar 27, 2021
    9. zooba commented on Apr 22, 2021

      @zooba
      Member

      New changeset 4696f12 by Steve Dower in branch 'master':
      bpo-35306: Avoid raising OSError from pathlib.Path.exists when passed an invalid filename (GH-25529)
      4696f12

    10. self-assigned this
      on Apr 22, 2021
    11. miss-islington commented on Apr 22, 2021

      @miss-islington
      Contributor

      New changeset 1575ea0 by Miss Islington (bot) in branch '3.8':
      bpo-35306: Avoid raising OSError from pathlib.Path.exists when passed an invalid filename (GH-25529)
      1575ea0

    12. miss-islington commented on Apr 22, 2021

      @miss-islington
      Contributor

      New changeset ab5d78c by Miss Islington (bot) in branch '3.9':
      bpo-35306: Avoid raising OSError from pathlib.Path.exists when passed an invalid filename (GH-25529)
      ab5d78c

    13. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    3.10 (EOL)end of life3.8 (EOL)end of life3.9 (EOL)end of lifeOS-windowsstdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions