Lib/importlib: Ensure stale empty python module directories don't break the build

Bug report

Bug description:

The Yocto project has been seeing mysterious build failures that were traced down to stale empty python module directories left behind by previous module versions:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14816

We are carrying the following patch to address the issue (it both ignores the empty directories, and looks at directories in deterministic sorted order):

--- a/Lib/importlib/metadata/__init__.py
+++ b/Lib/importlib/metadata/__init__.py
@@ -710,7 +710,14 @@ class Lookup:
         self.infos = FreezableDefaultDict(list)
         self.eggs = FreezableDefaultDict(list)
 
-        for child in path.children():
+        for child in sorted(path.children()):
+            childpath = pathlib.Path(path.root, child)
+            try:
+                if childpath.is_dir() and not any(childpath.iterdir()):
+                    # Empty directories aren't interesting
+                    continue
+            except PermissionError:
+                continue
             low = child.lower()
             if low.endswith((".dist-info", ".egg-info")):
                 # rpartition is faster than splitext and suitable for this purpose.

We'd like to discuss with upstream if such a fix is appropriate and will be taken as a proper pull request.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

gh-120492: Sync importlib_metadata 8.2.0 #124033

I would at least cross-post to https://github.com/python/importlib_metadata/issues given that primary development of importlib.metadata happens there.

This issue will be addressed by syncing with the latest importlib_metadata.

Sync with importlib_metadata 8.2.0

Another fun edgecase: if a .dist-info directory exists but is empty it gets returned by importlib.metadata.distributions() but as there is no metadata you end up with lots of Nones:

* Getting build dependencies for wheel...
Traceback (most recent call last):
  File "/work/ross/build/tmp/work/aarch64-linux/python3-meson-python-native/0.17.1/recipe-sysroot-native/usr/lib/python3.13/site-packages/build/__main__.py", line 178, in _handle_build_error
    yield
  File "/work/ross/build/tmp/work/aarch64-linux/python3-meson-python-native/0.17.1/recipe-sysroot-native/usr/lib/python3.13/site-packages/build/__main__.py", line 429, in main
    built = build_call(
        args.srcdir,
    ...<5 lines>...
        args.installer,
    )
  File "/work/ross/build/tmp/work/aarch64-linux/python3-meson-python-native/0.17.1/recipe-sysroot-native/usr/lib/python3.13/site-packages/build/__main__.py", line 238, in build_package
    out = _build(isolation, srcdir, outdir, distribution, config_settings, skip_dependency_check, installer)
  File "/work/ross/build/tmp/work/aarch64-linux/python3-meson-python-native/0.17.1/recipe-sysroot-native/usr/lib/python3.13/site-packages/build/__main__.py", line 172, in _build
    return _build_in_current_env(srcdir, outdir, distribution, config_settings, skip_dependency_check)
  File "/work/ross/build/tmp/work/aarch64-linux/python3-meson-python-native/0.17.1/recipe-sysroot-native/usr/lib/python3.13/site-packages/build/__main__.py", line 151, in _build_in_current_env
    missing = builder.check_dependencies(distribution, config_settings or {})
  File "/work/ross/build/tmp/work/aarch64-linux/python3-meson-python-native/0.17.1/recipe-sysroot-native/usr/lib/python3.13/site-packages/build/_builder.py", line 243, in check_dependencies
    return {u for d in dependencies for u in check_dependency(d)}
                                             ~~~~~~~~~~~~~~~~^^^
  File "/work/ross/build/tmp/work/aarch64-linux/python3-meson-python-native/0.17.1/recipe-sysroot-native/usr/lib/python3.13/site-packages/build/_util.py", line 53, in check_dependency
    if req.specifier and not req.specifier.contains(dist.version, prereleases=True):
                             ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/ross/build/tmp/work/aarch64-linux/python3-meson-python-native/0.17.1/recipe-sysroot-native/usr/lib/python3.13/site-packages/packaging/specifiers.py", line 917, in contains
    item = Version(item)
  File "/work/ross/build/tmp/work/aarch64-linux/python3-meson-python-native/0.17.1/recipe-sysroot-native/usr/lib/python3.13/site-packages/packaging/version.py", line 200, in __init__
    match = self._regex.search(version)
TypeError: expected string or bytes-like object, got 'NoneType'

Should importlib.metadata.distributions return a Distribution object for empty .dist-info directories?

Should importlib.metadata.distributions return a Distribution object for empty .dist-info directories?

My instinct - yes. The .dist-info directory is the primary indicator for the distribution existing. So even if it's not a valid one, it is present and thus should be reflected in the object model.

Merge branch 'main' into pythongh-120492/importlib-metadata-8.2.0

gh-120492: Sync importlib_metadata 8.2.0 (#124033)

With the merged PR, this issue can be closed.

The fix is in Python 3.15. Users requiring the future behavior on older Pythons should use importlib_metadata >= 8.2.

pythongh-120492: Sync importlib_metadata 8.2.0 (python#124033)

Navigation Menu

Repository navigation

Lib/importlib: Ensure stale empty python module directories don't break the build #120492

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Lib/importlib: Ensure stale empty python module directories don't break the build #120492

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

eli-schwartz commented on Jun 18, 2024

jaraco commented on Sep 12, 2024

rossburton commented on Apr 29, 2025

jaraco commented on Aug 14, 2025

jaraco commented on Aug 15, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions