Revisit adding lexical normalization support to pathlib

Feature or enhancement

Proposal:

I'd like to add a resolve_lexical method to concrete pathlib objects:

def resolve_lexical(self, /, strict=False):
    """Make the path absolute, and also normalize it, *without* resolving symlinks."""

As with resolve(), if strict is True and any segment of the given path doesn't exist, FileNotFoundError is raised (so /some/dir/_nonexistent_/.. would fail). If strict is False (the default), all path segments are processed without checking whether they exist (so /some/dir/_nonexistent_/.. lexically resolves to /some/dir).

While theoretically this could be added to PurePath without the strict option, I don't see any significant benefit to that (whereas I do see benefits to paralleling the Path.resolve() API as closely as possible).

As a minor note, adding this method would give a more direct way of checking if a path contains any symlinks at any level: path.resolve() == path.resolve_lexical() (vs the current path.is_symlink() or any(segment.is_symlink() for segment in path.parents)).

Chaining the two resolution methods would also be valid (path.resolve().resolve_lexical()), with symlinks then being resolved in the segments that actually exist, and the rest of the path, if any, being resolved lexically)

On my current project, I recently ran into a pair of subtle symlink-and-relative-path-handling bugs.

the original code used path.resolve() to fully resolve a path to its actual target. This gave a dynamic loading error on macOS, because one of the dynamic libraries the referenced executable needed was stored relative to the symlink, not relative to the actual binary. While that feels like a bug in the way the offending executable was packaged, handling it meant actively avoiding fully resolving paths, and instead respecting their nominal locations.
switching to path.absolute() not only turned off the symlink resolution, it also turned off the path normalisation that removes \..\ segments. This also resulted in dynamic loading errors on macOS when those segments were present in the executable reference (probably due to an underlying Python or OS API conditionally doing its own equivalent of path.resolve() when /../ was present in the path, since the resulting dynamic loading errors looked very similar to those I saw when hitting the first bug).

The resolution that handled both situations ended up being to use os.path to perform lexical normalisation (via os.path.abspath):

import os
import os.path

from pathlib import Path

def as_normalized_path(path:str|os.PathLike[str], /) -> Path:
    """Normalize given path and make it absolute, *without* resolving symlinks

    Expands user directory references, but *not* environment variable references.
    """
    # Ensure user directory references are handled as absolute paths
    expanded_path = os.path.expanduser(path)
    return Path(os.path.abspath(expanded_path))

Having to drop down to the lower level API to request "resolve /../ relative to the path as given" instead of the default "resolve /../ relative to symlink targets" feels like an unnecessary gap in the abstraction layer.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

Previously suggested here: #83105

When it comes to symlink security vulnerabilities arising from parent directory traversal, they mostly relate to using symlink resolution to reach an unexpected location:

a human reading /some/dir/symlink/../sibling will expect it to refer to /some/dir/sibling (lexical normalization)
Path(/some/dir/symlink/../sibling).resolve() actually refers to /parent_of_symlink_target/sibling (i.e. you have no idea where it points without access to the filesystem state that specifies the destination of /some/dir/symlink)

As a result, the rationale for rejection doesn't feel strong to me (since the intuitive behaviour is unavailable in the high level API, and instead only the subtle system state dependent behaviour is offered)

Also noting that Java does offer lexical normalisation on its Path abstraction: https://docs.oracle.com/en/java/javase/23/docs/api/java.base/java/nio/file/Path.html#normalize()

Linked PRs

gh-124825: Adding PurePath.erase_parents() to pathlib #135406

What is the use case for strict=True in context of lexical normalization?

If you want to rule out cases like /some/dir/_nonexistent_/.., that's hard to do without checking the filesystem as you go.

I don't mind leaving out strict=True though.

I only included it in my suggested API because Path.resolve() defines it (making it a smaller code transformation to switch between the two resolution methods), there is a sensible meaning for it (even in lexical resolution), and I don't think it would substantially add to the maintenance burden for the feature.

Edit: even if we leave out strict=True, I think the implicit Path.absolute() invocation still means this method should only exist on the concrete Path classes, and not on PurePath.

As a result, the rationale for rejection doesn't feel strong to me (since the intuitive behaviour is unavailable in the high level API, and instead only the subtle system state dependent behaviour is offered)

Opinions differ on this point! e.g. PEP 428 says:

Sane behaviour

Little of the functionality from os.path is reused. Many os.path functions are tied by backwards compatibility to confusing or plain wrong behaviour (for example, the fact that os.path.abspath() simplifies ".." path components without resolving symlinks first).

Ah, I knew I had seen a written objection to lexical normalisation somewhere, but didn't think to check the original PEP!

The context where this came up for me is symlinked virtual environments: fully resolving sys.executable inadvertently breaks you out of the venv, whereas lexical resolution respects the facade being presented to the application.

I think (but haven't confirmed), there are also some symlink games happening in the python-build-standalone macOS builds.

I think I'd be OK with adding this as a PurePath method that throws ValueError when given a non-absolute path, which is similar to PurePath.as_uri().

Users of Path objects could do path.absolute().normalized(), which is more wordy, but I think that's fine as they almost always want resolve() or absolute() rather than normalized().

I don't like a name involving "resolve" because I think it could imply symlink resolution, which isn't what we're doing. Nor are we using the POSIX algorithm for pathname resolution. Adding "lexical" helps, but I think there's still a small potential for confusion. Maybe.

Names involving "normalize" are a tiny bit problematic because pathlib already performs path normalization (specifically, removal of redundant slashes and . segments).

@barneygale

@barneygale wrote:

I think I'd be OK with adding this as a PurePath method … which is similar to PurePath.as_uri().

This is my preference as well. It's very simple to implement it this way. I've actually been monkey-patching this into a PurePath.normalize method for a while:

def _normalize(self):
    """
    Normalize a path (removing unnecessary '..')
    without attempting to resolve symlinks.
    """
    return self.__class__(self._flavour.normpath(self))
PurePath.normalize = _normalize

Allows me to do things like Path("/foo/bar/wherever", "../../whatever").normalize() → Path("/foo/whatever").

that throws ValueError when given a non-absolute path,

Why should it refuse to operate on relative paths? What's the downside to being able to do…

Path("foo/bar/wherever", "../../whatever").normalize() → Path("foo/whatever")
or even Path("wherever", "../../whatever").normalize() → Path("../whatever")

I don't like a name involving "resolve" because I think it could imply symlink resolution, which isn't what we're doing. Nor are we using the POSIX algorithm for pathname resolution. Adding "lexical" helps, but I think there's still a small potential for confusion. Maybe.

Agreed.

Names involving "normalize" are a tiny bit problematic because pathlib already performs path normalization (specifically, removal of redundant slashes and . segments).

In the spirit of "explicit is better than implicit", how about erase_parents()?

def erase_parents(self):
    r"""
    Remove path components referring to parent directories ('..').
    Unlike .resolve(), this occurs without an attempt to verify
    the existence or structure of any path components.

    If this is a relative path object, there may be leftover '..'
    segments after erasing parent segments.
    For example:

    - Path('spam/../../eggs').erase_parents() == Path('../eggs').

    If this is an absolute path, and it reaches the root filesystem
    (including the root of a drive or UNC share on Windows),
    further '..' components will be ignored.
    For example:

    - Path('/spam/../../../eggs').erase_parents() == Path('/eggs')
    - PureWindowsPath(r'c:\spam\..\..\eggs').erase_parents() == PureWindowsPath(r'c:\eggs')
    - PureWindowsPath(r'\\server\share\foo\..\..\eggs').erase_parents() == PureWindowsPath(r'\\server\share\eggs')
    """
    return self.__class__(self._flavour.normpath(self))

pythongh-124825: Adding PurePath.erase_parents() to pathlib

I proposed this in #135406.

It is currently targeting 3.12, and I'm not entirely sure how to update it for 3.13+, where the use of ntpath/posixpath functions in pathlib seems to be discouraged 🤔. Suggestions welcome.

@dlenski

@dlenski Did you retry to submit your addition? Only main is accepted as target branch.

I'm not entirely sure how to update it for 3.13+, where the use of ntpath/posixpath functions in pathlib seems to be discouraged 🤔. Suggestions welcome.

3.13 added PurePath.parser to refer to the underlying ntpath/posixpath module, so using that as appropriate instead of referencing ntpath or posixpath directly is likely to address the implementation issues you were facing.

Many times the motivation for this need is that equality doesn't work if one path is normalised and other isn't.

That is,

>>> Path("/something/not-here/../here") == Path("/something/here/")
False

This is no different than a wrapper over comparing str(path).

Since the usage of this library is object oriented, the equality should not matter on the exact string since different sources can give the path in different styles / ways.

Many times the motivation for this need is that equality doesn't work if one path is normalised and other isn't.

For checking if two paths point to the same file on the current machine, https://docs.python.org/3/library/pathlib.html#pathlib.Path.samefile is a better option than comparing the paths for equality.

Semantically, a non-normalised path is not the same as a normalised one (specifically because of the way stepping up the directory tree works if you've descended into a resolved symlink), hence equality checks considering them to be different.

Navigation Menu

Repository navigation

Revisit adding lexical normalization support to `pathlib` #124825

Feature or enhancement

Proposal:

Has this already been discussed elsewhere?

Links to previous discussion of this feature:

Linked PRs

Sane behaviour

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Revisit adding lexical normalization support to pathlib #124825

Description

Feature or enhancement

Proposal:

Has this already been discussed elsewhere?

Links to previous discussion of this feature:

Linked PRs

Activity

serhiy-storchaka commented on Oct 1, 2024

ncoghlan commented on Oct 1, 2024

barneygale commented on Oct 1, 2024

Sane behaviour

ncoghlan commented on Oct 2, 2024

barneygale commented on Oct 3, 2024

dlenski commented on Jun 11, 2025

dlenski commented on Jun 11, 2025

reneleonhardt commented on Jul 13, 2025

ncoghlan commented on Sep 30, 2025

siddhpant commented on Apr 30, 2026

ncoghlan commented on May 1, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Revisit adding lexical normalization support to `pathlib` #124825