Repository navigation
Regression of os.scandir() typecheck #11964
Description
Bug Report
Type variable inference related with os.scandir() seems to generate false-positives.
To Reproduce
Here is a minimal reproduction example, which is accepted by mypy 0.910 but rejected by 0.930 and 0.931.
from __future__ import annotations
import os
from pathlib import Path
def _calc_usage(target_path: Path | os.DirEntry) -> None:
with os.scandir(target_path) as scanner:
for entry in scanner:
if entry.is_dir():
_calc_usage(target_path)
else:
print('f')
_calc_usage(Path('.'))Expected Behavior
Success: no issues found in 1 source file
Actual Behavior
test.py:8: error: Value of type variable "AnyStr" of "scandir" cannot be "Union[DirEntry[Any], Any]"
Your Environment
- Mypy version used: 0.931
- Mypy command-line flags:
python -m mypy test.py - Mypy configuration options from
mypy.ini(and other config files): - Python version used: 3.9.6
- Operating system and version: macOS 12.1
Activity
JelleZijlstra commentedon Jan 10, 2022
For what it's worth, the only significant recent change to the stubs for scandir was to use
|syntax: https://github.com/python/typeshed/blame/master/stdlib/os/__init__.pyi#L711ktbarrett commentedon Jan 10, 2022
@achimnol Are you certain the original code is valid? typeshed does not mention that
os.scandirtakes aos.DirEntry.JelleZijlstra commentedon Jan 10, 2022
@ktbarrett
DirEntryis anos.PathLike.ktbarrett commentedon Jan 10, 2022
The change from
Unionto|did not affect anything. Confirmed it broke in 0.920.Changing the signature of scandir to the following works.
Changing the signature of AnyStr to use a Union bound also works. So the problem must be with how listed types are checked between 0.910 and 0.920.
ktbarrett commentedon Jan 10, 2022
git bisectshows 7808e82 (#10887).ktbarrett commentedon Jan 11, 2022
Started debugging a simpler example against the passing example below it that uses a Union bound instead of a list.
Fails:
Passes:
After much debugging (which is not easy at all because of how coarse-grained the switch on incremental compile is) I found that the difference was in how the type variable was inferred. Example 1 inferred the TypeVar as
Union[os.DirEntry[Any], Any]and Example 2 asAny. After debugging some more I discovered that only TypeVar bounds, and not the list, are considered in the below case, which ends up deciding the TypeVar constraints for the call.mypy/mypy/constraints.py
Lines 198 to 202 in 48d810d
achimnol commentedon Feb 16, 2022
How is this going? Just checking in. 😉
ktbarrett commentedon Feb 16, 2022
I spent a couple hours looking into it and I think it's due to a lack of support for type lists in
infer_constraints. It's not a trivial fix.