Repository navigation
stubtest crash when a referenced module is a DLL #13822
Copy link
Copy link
Closed
Labels
Description
Crash Report
stubtest crashes when a referenced module is a DLL, as it seemingly tries to parse the .dll's content as text.
Traceback
PS C:\Users\Avasam\Documents\Git\typeshed> python .\tests\stubtest_third_party.py pywin32
pywin32... fail
C:\Users\Avasam\AppData\Local\Temp\tmp3yxfa81_\Scripts\pip.exe install pywin32[]==304.* mypy==0.982
MYPYPATH=C:\Users\Avasam\Documents\Git\typeshed\stubs\pywin32 C:\Users\Avasam\AppData\Local\Temp\tmp3yxfa81_\Scripts\python.exe -m mypy.stubtest --custom-typeshed-dir C:\Users\Avasam\Documents\Git\typeshed pythonwin win32 win32comext pythoncom _win32typing --allowlist C:\Users\Avasam\Documents\Git\typeshed\stubs\pywin32\@tests\stubtest_allowlist.txt
Traceback (most recent call last):
File "C:\Program Files\Python39\lib\tokenize.py", line 330, in find_cookie
line_string = line.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x90 in position 2: invalid start byte
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files\Python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Program Files\Python39\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\Avasam\AppData\Local\Temp\tmp3yxfa81_\lib\site-packages\mypy\stubtest.py", line 1713, in <module>
sys.exit(main())
File "C:\Users\Avasam\AppData\Local\Temp\tmp3yxfa81_\lib\site-packages\mypy\stubtest.py", line 1709, in main
return test_stubs(parse_options(sys.argv[1:]))
File "C:\Users\Avasam\AppData\Local\Temp\tmp3yxfa81_\lib\site-packages\mypy\stubtest.py", line 1605, in test_stubs
print(error.get_description(concise=args.concise))
File "C:\Users\Avasam\AppData\Local\Temp\tmp3yxfa81_\lib\site-packages\mypy\stubtest.py", line 138, in get_description
runtime_line = inspect.getsourcelines(self.runtime_object)[1]
File "C:\Program Files\Python39\lib\inspect.py", line 1006, in getsourcelines
lines, lnum = findsource(object)
File "C:\Program Files\Python39\lib\inspect.py", line 831, in findsource
lines = linecache.getlines(file, module.__dict__)
File "C:\Program Files\Python39\lib\linecache.py", line 46, in getlines
return updatecache(filename, module_globals)
File "C:\Program Files\Python39\lib\linecache.py", line 136, in updatecache
with tokenize.open(fullname) as fp:
File "C:\Program Files\Python39\lib\tokenize.py", line 394, in open
encoding, lines = detect_encoding(buffer.readline)
File "C:\Program Files\Python39\lib\tokenize.py", line 371, in detect_encoding
encoding = find_cookie(first)
File "C:\Program Files\Python39\lib\tokenize.py", line 335, in find_cookie
raise SyntaxError(msg)
SyntaxError: invalid or missing encoding declaration for 'C:\\Users\\Avasam\\AppData\\Local\\Temp\\tmp3yxfa81_\\lib\\site-packages\\pywin32_system32\\pywintypes39.dll'
To Reproduce
The easiest way to reproduce would be to use the pywin32 stubs from typeshed as an example. It has references to pywintypes39.dll and pythoncom39.dll. Make sure that METADATA.toml has:
[tool.stubtest]
ignore_missing_stub = false
skip = falsepython .\tests\stubtest_third_party.py pywin32
Your Environment
- Mypy version used: 0.981 (compiled: yes)
- Mypy command-line flags: (see above)
- Mypy configuration options from
mypy.ini(and other config files): (see above) - Python version used: 3.9.13
- Operating system and version: Windows 10.0.19044 Build 19044
Reactions are currently unavailable
Activity
Avasam commentedon Oct 5, 2022
Related, very similar error in stubgen as well:
stubgen -p pythoncomResults in:
Critical error during semantic analysis: mypy: can't decode file '[...]\.venv\Lib\site-packages\pywin32_system32\pythoncom39.dll': 'utf-8' codec can't decode byte 0x90 in position 2: invalid start byteJelleZijlstra commentedon Oct 5, 2022
This feels like a CPython bug, since stubtest is just calling
inspect.getsourcelines. I think the relevant lines are here: https://github.com/python/cpython/blob/b44372e03c5461b6ad3d89763a9eb6cb82df07a4/Lib/inspect.py#L946Basically, inspect doesn't try to find a source file if the object's
__file__has one of theimportlib.machinery.EXTENSION_SUFFIXES. But in this case, I suppose pywin32 exposes an object with a__file__that points to a .dll.Where should this be fixed?
.pyextension files.__file__pointing to a file that's not a valid extension module.getsourcelines()failing. That's not very principled, but it's good for stubtest to be resilient to odd failures, so I think we should do that.stubtest: catch SyntaxError from inspect.getsourcelines
stubtest: catch SyntaxError from inspect.getsourcelines (#13848)