Skip to content

Make .test parser friendlier - #15385

Merged
hauntsaninja merged 3 commits into
python:masterfrom
ikonst:testcase-parse-gotchas
Jun 7, 2023
Merged

hauntsaninja merged 3 commits into
python:masterfrom
ikonst:testcase-parse-gotchas

Conversation

@ikonst

@ikonst ikonst commented Jun 7, 2023 •

Copy link
Copy Markdown
Contributor

Quality of life improvements:

  • Consolidate parser messages in a common function. We're using pytest.fail(..., pytrace=False) to eliminate stack trace noise.
  • Make sure error messages have correct line offsets, that refer to the global line number in the file (that you can scroll to with an editor). Previously we used TestItem.line which is relative to testcase (e.g. for main item of testcase starting at line 1234, it'll be 1).
  • Always parse case sections, erroring out on invalid names; previously invalid [case]s were ignored during split_test_cases but then resurfaced when parsing (as they'd end up being a section of the preceding case):
    [case abc]
    ...
    [case something-invalid]  <-- skipped during split_test_cases, raised as invalid section of abc
    ...
    
  • Remove the assertion in teardown, since it would invariably (and noisily!) trip up whenever setup failed.

I've added some rudimentary tests for this, also moving the --update-data test along into mypy/tests/meta. Since those are "meta tests", we shouldn't make them comprehensive, only trying to keep up some basic level of contributor ergonomics.

@ikonst
ikonst force-pushed the testcase-parse-gotchas branch from cbf5f72 to 5e2c7eb Compare June 7, 2023 02:20
@ikonst ikonst changed the title Fix testcase parsing gotchas Make .test parser friendlier Jun 7, 2023
Comment thread mypy/test/data.py
self.steps = [steps.get(num, []) for num in range(2, max_step + 1)]

def teardown(self) -> None:
assert self.old_cwd is not None and self.tmpdir is not None, "test was not properly set up"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would trip up whenever setup failed, adding to the noise (and making the setup failure, which is the underlying cause, harder to find)

Comment thread mypy/test/data.py
Comment on lines -650 to -656
r"^\[case ([a-zA-Z_0-9]+)"
r"(-writescache)?"
r"(-only_when_cache|-only_when_nocache)?"
r"(-posix|-windows)?"
r"(-skip)?"
r"(-xfail)?"
r"\][ \t]*$\n",

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously, when [case foo-bar-baz] didn't comply with this pattern, we'd simply skip it and it would end up merging into the section of the preceding [case].

@ikonst
ikonst force-pushed the testcase-parse-gotchas branch from ac15b53 to 48eb214 Compare June 7, 2023 14:11
@ikonst
ikonst force-pushed the testcase-parse-gotchas branch from 48eb214 to b39cd0f Compare June 7, 2023 14:13

@hauntsaninja hauntsaninja left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@hauntsaninja
hauntsaninja merged commit fe0714a into python:master Jun 7, 2023
@ikonst
ikonst deleted the testcase-parse-gotchas branch June 7, 2023 20:12
@cdce8p

cdce8p commented Jun 12, 2023

Copy link
Copy Markdown
Collaborator

It seems as though this PR again broke the mypy wheel build. Maybe a similar issue to the one fixed in #15383?
https://github.com/mypyc/mypy_mypyc-wheels/actions/runs/5204230354/jobs/9388182425#step:4:7209

JelleZijlstra pushed a commit that referenced this pull request Jun 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants