[dev] wrong-version version tests disappear completely, but should probably skip instead

Consider the following:

uv run runtests.py testTemplateString
usage: runtests.py [self] [self-packaging] [lint] [pytest-fast] [pytest-cmdline] [pytest-slow] [mypyc-fast] [pytest-extra] [mypyc-extra] [names of individual tests and files...]

Run the given tests. If given no arguments, run everything except pytest-extra and mypyc-extra. Unrecognized arguments will be interpreted as individual test names / substring expressions (or, if they end in .test, individual test files) and this script will try to run them.
run testTemplateString: ['pytest', '-n0', '-k', 'testTemplateString']
========================================================================================================== test session starts ===========================================================================================================
platform win32 -- Python 3.13.7, pytest-8.4.2, pluggy-1.6.0
rootdir: C:\Users\wyatt\files\gits\wyattscarpenter\cfork\mypy
configfile: pyproject.toml
testpaths: mypy/test, mypyc/test
plugins: cov-7.0.0, xdist-3.8.0
collected 13431 items / 13431 deselected / 0 selected

======================================================================================================= 13431 deselected in 1.78s ========================================================================================================

FAILED: testTemplateString

This is a real test, in parse-python314.test (and also, oddly enough, in check-python314.test — not sure what's up with that) but, apparently, because my python version is <3.14, it disappears. It should probably pytest skip instead (yellow "s" on the list of results), that seems like the correct result semantically, and is less mysterious than the test just not being there.

A good place to start investigating this change might be /mypy/test/testparse.py.

Just replace the discovery patch with pytest.skip() in TypeCheckSuite.run_case, I guess?

mypy/mypy/test/testcheck.py

Lines 41 to 60 in 985dc7f

    
           # Tests that use Python version specific features: 
        
           if sys.version_info < (3, 11): 
        
               typecheck_files.remove("check-python311.test") 
        
           if sys.version_info < (3, 12): 
        
               typecheck_files.remove("check-python312.test") 
        
           if sys.version_info < (3, 13): 
        
               typecheck_files.remove("check-python313.test") 
        
           if sys.version_info < (3, 14): 
        
               typecheck_files.remove("check-python314.test") 
        
           class TypeCheckSuite(DataSuite): 
        
               files = typecheck_files 
        
               def run_case(self, testcase: DataDrivenTestCase) -> None: 
        
                   if os.path.basename(testcase.file) == "check-modules-case.test": 
        
                       with tempfile.NamedTemporaryFile(prefix="test", dir=".") as temp_file: 
        
                           temp_path = Path(temp_file.name) 
        
                           if not temp_path.with_name(temp_path.name.upper()).exists(): 
        
                               pytest.skip("File system is not case‐insensitive")

Navigation Menu

Repository navigation

[dev] wrong-version version tests disappear completely, but should probably skip instead #20398

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	# Tests that use Python version specific features:
	if sys.version_info < (3, 11):
	typecheck_files.remove("check-python311.test")
	if sys.version_info < (3, 12):
	typecheck_files.remove("check-python312.test")
	if sys.version_info < (3, 13):
	typecheck_files.remove("check-python313.test")
	if sys.version_info < (3, 14):
	typecheck_files.remove("check-python314.test")


	class TypeCheckSuite(DataSuite):
	files = typecheck_files

	def run_case(self, testcase: DataDrivenTestCase) -> None:
	if os.path.basename(testcase.file) == "check-modules-case.test":
	with tempfile.NamedTemporaryFile(prefix="test", dir=".") as temp_file:
	temp_path = Path(temp_file.name)
	if not temp_path.with_name(temp_path.name.upper()).exists():
	pytest.skip("File system is not case‐insensitive")

[dev] wrong-version version tests disappear completely, but should probably skip instead #20398

Description

Activity

sterliakov commented on Dec 11, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions