Single threaded pytest spuriously fails for testcmdline and testpythoneval #4127

Activity

1. elazarg commented on Oct 17, 2017

   

   elazarg

   on Oct 17, 2017

   ContributorAuthor

   More actions

   The following patch fixed the problem for me, but I can't reproduce the more serious issue, so it might not be the right solution (this diff is only about testcmdline; pythoneval should be similar):

   diff --git a/mypy/test/testcmdline.py b/mypy/test/testcmdline.py
   index 26f1780..ede8f30 100644
   --- a/mypy/test/testcmdline.py
   +++ b/mypy/test/testcmdline.py
   @@ -12,7 +12,7 @@ import sys
    from typing import Tuple, List, Dict, Set
   
    from mypy.myunit import Suite, SkipTestCaseException, AssertionFailure
   -from mypy.test.config import test_data_prefix, test_temp_dir
   +from mypy.test.config import test_data_prefix, test_temp_dir, PREFIX
    from mypy.test.data import fix_cobertura_filename
    from mypy.test.data import parse_test_cases, DataDrivenTestCase, DataSuite
    from mypy.test.helpers import assert_string_arrays_equal, normalize_error_messages
   @@ -58,10 +58,13 @@ def test_python_evaluation(testcase: DataDrivenTestCase) -> None:
        # Type check the program.
        fixed = [python3_path,
                 os.path.join(testcase.old_cwd, 'scripts', 'mypy')]
   +    env = os.environ.copy()
   +    env['PYTHONPATH'] = PREFIX
        process = subprocess.Popen(fixed + args,
                                   stdout=subprocess.PIPE,
                                   stderr=subprocess.STDOUT,
   -                               cwd=test_temp_dir)
   +                               cwd=test_temp_dir,
   +                               env=env)
        outb = process.stdout.read()
        # Split output into lines.
        out = [s.rstrip('\n\r') for s in str(outb, 'utf8').splitlines()]
   
2. gvanrossum commented on Oct 17, 2017

   

   gvanrossum

   on Oct 17, 2017

   Member

   More actions

   Thanks! I'll try it when I have more time. It may miraculously work.
3. gvanrossum commented on Oct 17, 2017

   

   gvanrossum

   on Oct 17, 2017

   Member

   More actions

   Yeah, this fixes it for me. I tried pythoneval.py as well and it passes with -n0 -- so it doesn't need this fix. This is because it doesn't fork a subprocess to run mypy -- it uses mypy.api.run(). Perhaps testcmdline.py could be modified to do that too? I don't think we gain much by forking a process. (Though we don't seem to lose much either.)
4. emmatyping commented on Oct 17, 2017

   

   emmatyping

   on Oct 17, 2017

   Member

   More actions

   When I moved pythoneval to mypy.api.run() I also considered moving testcmdline.py to it as well but did not, as I worried about giving up testing the actual OS<->mypy interaction. On the other hand, I feel that it hasn't really ever been an issue, and there is not much that can go wrong without Python itself being broken.
5. gvanrossum commented on Oct 17, 2017

   

   gvanrossum

   on Oct 17, 2017 via email

   Member

   More actions

   Maybe we can include 1-2 separate tests of the actual OS<->mypy interaction; one for a passing program and one for a failing program. (Maybe a few more to exercise the exceptional workflows that raise CompileError prematurely, e.g. missing file or syntax errors.)
6. ilevkivskyi

   added

   topic-tests

   on May 18, 2018
7. msullivan

   added a commit that references this issue on May 25, 2018

   Fix testcmdline when it is run with py.test -n0

   ...

   69dc33a
8. gvanrossum

   closed this as completedin #5113on May 26, 2018
9. gvanrossum

   added a commit that references this issue on May 26, 2018

   Fix testcmdline when it is run with py.test -n0 (#5113)

   ...

   7eebb8f