Skip to content

Pytest discovery fails #21576

Closed as not planned
Closed as not planned
Pytest discovery fails#21576
Assignees
eleanorjboyd
Labels
area-testingbugIssue identified by VS Code Team member as probable buginfo-neededIssue requires more information from postertriage-neededNeeds assignment to the proper sub-team

Description

@kjohnsen
Kyle Johnsen (kjohnsen)
opened on Jul 6, 2023

Type: Bug

Behaviour

Expected vs. Actual

I expect discovery to work, for the explorer to populate with tests. Instead I get pytest discovery error (see Output > Python)

Steps to reproduce:

  1. Click the 'refresh tests' button on the testing pane

Diagnostic data

  • Python version (& distribution if applicable, e.g. Anaconda): 3.10.9
  • Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Conda
  • Value of the python.languageServer setting: Pylance
Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python) 

2023-07-06 19:06:05.076 [info] Discover tests for workspace name: cleo - uri: /home/kyle/dbgt/projects/cleo
2023-07-06 19:06:05.076 [info] Running discovery for pytest using the new test adapter.
2023-07-06 19:06:05.093 [info] > conda run -n cleo10 --no-capture-output python ~/.vscode-server-insiders/extensions/ms-python.python-2023.11.11801006/pythonFiles/get_output_via_markers.py -m pytest -p vscode_pytest --collect-only tests
2023-07-06 19:06:05.093 [info] cwd: .
2023-07-06 19:06:07.783 [info] Test server connected to a client.
2023-07-06 19:06:07.785 [info] Using result resolver for discovery
2023-07-06 19:06:07.785 [error] pytest test discovery error 
 Error Occurred, traceback: Traceback (most recent call last):
  File "/home/kyle/.vscode-server-insiders/extensions/ms-python.python-2023.11.11801006/pythonFiles/vscode_pytest/__init__.py", line 218, in pytest_sessionfinish
    session_node: Union[TestNode, None] = build_test_tree(session)
  File "/home/kyle/.vscode-server-insiders/extensions/ms-python.python-2023.11.11801006/pythonFiles/vscode_pytest/__init__.py", line 259, in build_test_tree
    session_node = create_session_node(session)
  File "/home/kyle/.vscode-server-insiders/extensions/ms-python.python-2023.11.11801006/pythonFiles/vscode_pytest/__init__.py", line 404, in create_session_node
    "path": session.path,
AttributeError: 'Session' object has no attribute 'path'

User Settings 


languageServer: "Pylance"

formatting
• provider: "none"

testing
• pytestArgs: "<placeholder>"
• pytestEnabled: true
• autoTestDiscoverOnSaveEnabled: false

Extension version: 2023.13.11871009
VS Code version: Code - Insiders 1.80.0-insider (5b7440441bbd2548a862f7321cafdfa1597132e5, 2023-06-30T21:12:14.732Z)
OS version: Windows_NT x64 10.0.22621
Modes:
Remote OS version: Linux x64 5.10.16.3-microsoft-standard-WSL2

System Info
Item Value
CPUs Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz (16 x 2400)
GPU Status 2d_canvas: enabled
canvas_oop_rasterization: disabled_off
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabled
Load (avg) undefined
Memory (System) 31.73GB (9.47GB free)
Process Argv --crash-reporter-id 13862219-8062-4bb8-969d-b654fdb7b5e1
Screen Reader no
VM 0%
Item Value
Remote WSL: Ubuntu
OS Linux x64 5.10.16.3-microsoft-standard-WSL2
CPUs Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz (16 x 2400)
Memory (System) 15.50GB (12.88GB free)
VM 0%
A/B Experiments 
vsliv695:30137379
vsins829:30139715
vsliv368:30146709
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
pythontb:30258533
pythonptprofiler:30281269
vshan820:30294714
vscod805cf:30301675
bridge0708:30335490
bridge0723:30353136
vsaa593cf:30376535
pythonvs932:30404738
py29gd2263:30776702
vsclangdf:30492506
c4g48928:30535728
dsvsc012cf:30540253
pynewext54:30618038
pylantcb52:30590116
pyind779:30611226
pythonsymbol12:30651887
showlangstatbar:30737417
pythonms35:30671666
57b77579:30687741
pythonfmttext:30716741
pythoncmvfstr:30726892
fixshowwlkth:30771523
showindicator:30766888
pythongtdpath:30726887
i26e3531:30780429
gsofb:30778559
pythonnosmt12:30779711
pythonidxpt:30768918
pythondjangots:30768917
pythonnoceb:30776497
copilotsettingt:30767686
e537b577:30772214
h0f32768:30779712
synctok:30783813
dsvsc013:30777762
dsvsc014:30777825

Activity

  1. github-actions
    added
    triage-neededNeeds assignment to the proper sub-team
    on Jul 6, 2023

  2. eleanorjboyd commented on Jul 10, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Jul 10, 2023
    Member

    Hi! Could you send your pytest version?

  3. Eleanor Boyd (eleanorjboyd)
    self-assigned this
    on Jul 10, 2023
  4. github-actions
    added
    info-neededIssue requires more information from poster
    on Jul 10, 2023
  5. Eleanor Boyd (eleanorjboyd)
    added
    bugIssue identified by VS Code Team member as probable bug
    area-testing
    and removed
    info-neededIssue requires more information from poster
    on Jul 10, 2023

  6. eleanorjboyd commented on Jul 10, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Jul 10, 2023
    Member

    likely it is #21540. If the version matches maybe the change didn't get out in the release.

  7. Eleanor Boyd (eleanorjboyd)
    added
    info-neededIssue requires more information from poster
    on Jul 10, 2023

  8. kjohnsen commented on Jul 10, 2023

    @kjohnsen
    Kyle Johnsen (kjohnsen)
    on Jul 10, 2023 via email
    Author
  9. github-actions
    removed
    info-neededIssue requires more information from poster
    on Jul 10, 2023

  10. eleanorjboyd commented on Jul 11, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Jul 11, 2023
    Member

    Hi! Looking into this now but if you upgrade your pytest version you should no longer experience this issue. Thanks

  11. kjohnsen commented on Jul 14, 2023

    @kjohnsen
    Kyle Johnsen (kjohnsen)
    on Jul 14, 2023
    Author

    I upgraded to 7.4 and it's still broken. This is what I get on Ouput->Python:

    2023-07-13 17:53:48.535 [info] Discover tests for workspace name: cleo - uri: /home/kyle/dbgt/projects/cleo
2023-07-13 17:53:48.535 [info] Running discovery for pytest using the new test adapter.
2023-07-13 17:53:48.591 [info] > conda run -n cleo --no-capture-output python ~/.vscode-server-insiders/extensions/ms-python.python-2023.13.11941005/pythonFiles/get_output_via_markers.py -m pytest -p vscode_pytest --collect-only tests
2023-07-13 17:53:48.592 [info] cwd: .
2023-07-13 17:53:57.031 [info] Test server connected to a client.
2023-07-13 17:53:57.059 [info] Using result resolver for discovery

    At least the test explorer populates, but when I click "refresh tests" I get an error notification Cannot read properties of undefined (reading 'test').

    When I try to run them using the test explorer, the test results pane pops up and says "Finished running tests!" but none have actually been run.

    On a side note, the Insiders test explorer seems to be broken a lot for Python...I usually use the command line for tests since it's broken so often. Even so, I'd like to be able to use the debugger more often.

  12. eleanorjboyd commented on Jul 17, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Jul 17, 2023
    Member

    Hi! The error notification is microsoft/vscode#187770 and should be fixed now. The result panel saying "Finished running tests" and not having them run is likely a different problem. Could you check your logs and see if an error is seen there.

    In the meantime, we are releasing a new infrastructure which could be your problem at this point in time as we are working out issues coming from users. This rewrite is helping with old buggy behavior but is still experimental. You can opt out of the rewrite as I get this fix in by setting this in your **user **settings: "python.experiments.optOutFrom": ["pythonTestAdapter"],. If this doesn't work let me know as this will mean it is a different issue. Thank you and I will send updates in this thread as I get a fix in.

  13. github-actions
    added
    info-neededIssue requires more information from poster
    on Jul 17, 2023

  14. kjohnsen commented on Jul 18, 2023

    @kjohnsen
    Kyle Johnsen (kjohnsen)
    on Jul 18, 2023
    Author

    Right. That error notification is now gone. But I still have "finished running tests" without tests being run. In Output->Python it says the new adapter is being used, and I don't see an error message in "Python", "Python Test Log", or "Python Language Server" when I try to run tests. Tests appear to be discovered correctly.

    When I opt out from the new test adapter, test discovery is broken. All that shows up is the tests folder with no tests in it.

  15. github-actions
    removed
    info-neededIssue requires more information from poster
    on Jul 18, 2023

  16. eleanorjboyd commented on Jul 18, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Jul 18, 2023
    Member

    understood- thanks for the info! Will investigate

  17. jonodhawkins commented on Jul 20, 2023

    @jonodhawkins
    Jonathan Hawkins (jonodhawkins)
    on Jul 20, 2023

    Hi,

    I am having the same issue described above:

    • "Finished Running Tests" without being run
    • No error messages in "Python", "Python Test Log", "Python Language Server"
    • Tests run successfully (and see output) when running pytest from terminal.

    I have:

    • opted in to ["pythonTestAdapter"]
    • freshly installed VS Code (removed local .config/.vscode configuration files), installation details below

    The project I was experiencing this with was stored on a second disk, which I opened in VSCode using a symbolic link to my home directory.

    Opened path (symlink): /home/jono/storage/[project_name]
Actual path: /mnt/storage/[project_name]

    I replicated a simple project (see below) on my local drive with only a tests directory and a virtual environment to test whether the issue was to do my with my project structure. The tests pass and the "Testing" panel UI is correctly updated.

    When I load the original project through its 'actual path' (i.e. not using the symlink, /mnt/storage/...) then the tests are also run and the UI is correctly updated.

    Perhaps this points to an issue with how the extension is resolving path names?

    Happy to provide any additional info.

    System Info

    Version: 1.80.1
Commit: 74f6148eb9ea00507ec113ec51c489d6ffb4b771
Date: 2023-07-12T17:22:25.257Z
Electron: 22.3.14
ElectronBuildId: 21893604
Chromium: 108.0.5359.215
Node.js: 16.17.1
V8: 10.8.168.25-electron.0
OS: Linux x64 5.19.0-46-generic

    Test Project Structure

    ./
    test/
        base/
            test_simple.py
    .venv/
        [installed via python3 -m venv .venv]
    .vscode/
        settings.json

    test_simple.py

    import pytest
def test_hello():
    assert "Hello".lower() == "hello"

    settings.json

    {
    "python.testing.pytestArgs": [
        "tests"
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.pytestEnabled": true
}

  18. eleanorjboyd commented on Aug 18, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Aug 18, 2023
    Member

    Hi Jonathan Hawkins (@jonodhawkins) thank you for the info! Looks like an issue with symlinks, which is something we know is broken. I am currently trying to fix it but having trouble repro-ing the behavior. To clarify your steps Jonathan Hawkins (@jonodhawkins),

    1. create "Test Project Structure" as described at a local path /mnt/storage/[project_name]
    2. create "test_simple.py" as described
    3. create symlink ln -s /mnt/storage/[project_name] /home/jono/storage/[project_name]
    4. cd to /home/jono/storage/[project_name]
    5. run code . to open vscode
    6. try to find tests and they are not being discovered

    Thanks for your help!

  19. github-actions
    added
    info-neededIssue requires more information from poster
    on Aug 18, 2023

  20. eleanorjboyd commented on Aug 18, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Aug 18, 2023
    Member

    there are a few different issues about testing and symlinks so I want to consolidate the discussion to this one: #21373 if possible. Thanks

  21. github-actions commented on Sep 18, 2023

    @github-actions
    github-actionsbot
    on Sep 18, 2023

    Because we have not heard back with the information we requested, we are closing this issue for now. If you are able to provide the info later on, then we will be happy to re-open this issue to pick up where we left off.

    Happy Coding!

  22. github-actions
    closed this as not plannedon Sep 18, 2023

  23. eleanorjboyd commented on Sep 18, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Sep 18, 2023
    Member

    Hi Jonathan Hawkins (@jonodhawkins) thank you for the info! Looks like an issue with symlinks, which is something we know is broken. I am currently trying to fix it but having trouble repro-ing the behavior. To clarify your steps Jonathan Hawkins (@jonodhawkins),

    1. create "Test Project Structure" as described at a local path /mnt/storage/[project_name]
    2. create "test_simple.py" as described
    3. create symlink ln -s /mnt/storage/[project_name] /home/jono/storage/[project_name]
    4. cd to /home/jono/storage/[project_name]
    5. run code . to open vscode
    6. try to find tests and they are not being discovered

    Thanks for your help!

    if anyone has repro steps please comment and I can reopen- thanks

  24. github-actions
    locked as resolved and limited conversation to collaborators on Oct 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area-testingbugIssue identified by VS Code Team member as probable buginfo-neededIssue requires more information from postertriage-neededNeeds assignment to the proper sub-team

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions