Repository navigation
Add support for unittest subtest with pytest as the adapter #24455
Description
Activity
Giving this a try, when I try it from the terminal I get this:
test_subtest.py::test [First subtest] SUBPASS [100%] test_subtest.py::test [Second subtest] SUBFAIL [100%] test_subtest.py::test PASSEDAny idea why
test_subtest.py::testis marked as passed if one of its subtests failed?create issue to discuss design in the pytest-subtest repo since another question came up as well. Feel free to comment in either place with your thoughts
pytest-dev/pytest-subtests#184Any idea why
test_subtest.py::testis marked as passed if one of its subtests failed?I can only confirm that the whole test is marked as PASSED. In the summary section it says that 2 [subtests] failed, 1 [test] passed, 3 subtests passed.
In VS Code test panel it shows the whole "test_x" as failed. And it is more appropriate than just ignoring subtests results as pythonTestAdapter does.
tests/integration/test_api.py::test_x [First subtest] SUBPASS tests/integration/test_api.py::test_x [Second subtest] SUBFAIL tests/integration/test_api.py::test_x [Third subtest] SUBFAIL tests/integration/test_api.py::test_x [Fourth subtest] SUBPASS tests/integration/test_api.py::test_x [Fifth subtest] SUBPASS tests/integration/test_api.py::test_x PASSED ===================================================================== FAILURES ===================================================================== _____________________________________________________________ test_x [Second subtest] ______________________________________________________________ subtests = SubTests(...) def test_x(subtests): with subtests.test(msg="First subtest"): assert 1 == 1 with subtests.test(msg="Second subtest"): > assert 2 == 1 E assert 2 == 1 tests\integration\test_api.py:185: AssertionError ---------------------------------------------------------------- Captured log call ----------------------------------------------------------------- ______________________________________________________________ test_x [Third subtest] ______________________________________________________________ subtests = SubTests(...) def test_x(subtests): with subtests.test(msg="First subtest"): assert 1 == 1 with subtests.test(msg="Second subtest"): assert 2 == 1 with subtests.test(msg="Third subtest"): > assert 3 == 1 E assert 3 == 1 tests\integration\test_api.py:187: AssertionError ---------------------------------------------------------------- Captured log call ----------------------------------------------------------------- ============================================================= short test summary info ============================================================== [Second subtest] SUBFAIL tests/integration/test_api.py::test_x - assert 2 == 1 [Third subtest] SUBFAIL tests/integration/test_api.py::test_x - assert 3 == 1 ========================================== 2 failed, 1 passed, 11 deselected, 3 subtests passed in 3.84s ===========================================In the recent versions of VS Code it is not possible to opt out from pythonTestAdapter? I want my subtests to keep failing:-)
I've tried"python.experiments.optOutFrom": [ "pythonTestAdapter" ],
but this doesn't help.
I'm getting a simmilar error.
My code:
import pytest def add(a, b): return a + b def test_subtests(subtests: pytest.Subtests): with subtests.test(msg="addition"): assert add(1, 1) == 2 with subtests.test(msg="error"): assert add(1, 1) == 5Console output:
Running pytest with args: ['-p', 'vscode_pytest', '--rootdir=h:\\python\\vc-subtests', 'h:\\python\\vc-subtests\\test_calculator.py::test_subtests'] ============================= test session starts ============================= platform win32 -- Python 3.13.0, pytest-9.0.1, pluggy-1.6.0 rootdir: h:\python\vc-subtests collected 1 item test_calculator.py uF [100%] ================================== FAILURES =================================== ____________________________ test_subtests [error] ____________________________ subtests = <_pytest.subtests.Subtests object at 0x000001DF030206E0> def test_subtests(subtests: pytest.Subtests): with subtests.test(msg="addition"): assert add(1, 1) == 2 with subtests.test(msg="error"): > assert add(1, 1) == 5 E assert 2 == 5 E + where 2 = add(1, 1) test_calculator.py:11: AssertionError ________________________________ test_subtests ________________________________ contains 1 failed subtest =========================== short test summary info =========================== SUBFAILED[error] test_calculator.py::test_subtests - assert 2 == 5 FAILED test_calculator.py::test_subtests - contains 1 failed subtest ============================== 2 failed in 0.09s ============================== Finished running tests!Yet, the testing panel shows "green".
VS Code version: Code 1.106.1 (cb1933bbc38d329b3595673a600fab5c7368f0a7, 2025-11-16T10:20:23.745Z)
OS version: Windows_NT x64 10.0.22631
Modes:Extensions (4)
Extension Author (truncated) Version debugpy ms- 2025.16.0 python ms- 2025.18.0 vscode-pylance ms- 2025.9.1 vscode-python-envs ms- 1.12.0 - changed the title
[-]The new pythonTestAdapter breaks python-subtests[/-][+]The new pythonTestAdapter breaks pytest subtests[/+]on Nov 18, 2025 Pytest has recently integrated pytest-subtests functionality into the core, but VS Code behaviour is still the same.
Reacted by Zak Kohlerthis is great that it is ported over! I will put this on the backlog but can also provide code pointers if someone would like to give it a try themselves!
Reacted by Zak Kohler- changed the title
[-]The new pythonTestAdapter breaks pytest subtests[/-][+]Add support for unittest subtest with pytest as the adapter[/+]on Nov 18, 2025 Thanks for the feature request! We are going to give the community 60 days from when this issue was created to provide 7 👍 upvotes on the opening comment to gauge general interest in this idea. If there's enough upvotes then we will consider this feature request in our future planning. If there's unfortunately not enough upvotes then we will close this issue.
This makes this extension much less useful, as the tests view doesn’t reliably mark failed tests as failed, which also affects the “rerun failed tests” button.
We immediately started using subtests heavily, but now we kind of have to use the CLI again to run tests until this is fixed.
Reacted by Makarov Andrey and Manuelpytest should still run the unittest subtest it just wont display each subtest in the UI. Are you seeing a different behavior than this?
- Last i checked, a failed subtest could still report back a green checkmark. depending on test order? or maybe always?…On Fri, Feb 27, 2026 at 11:59 AM Eleanor Boyd ***@***.***> wrote: *eleanorjboyd* left a comment (microsoft/vscode-python#24455) <#24455 (comment)> pytest should still run the unittest subtest it just wont display each subtest in the UI. Are you seeing a different behavior than this? — Reply to this email directly, view it on GitHub <#24455 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABORVLPZMJOLPLQS7BQZOHT4OBZXFAVCNFSM6AAAAACMO26FAWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTSNZUGAYTGMBTGA> . You are receiving this because you commented.Message ID: ***@***.***>
ok this would be a separate bug report than. This issue was about having the subtests show in the view when using pytest but that is a bug I would want to resolve. If you could file another issue I can take a look
Zak Kohler (@y2kbugger) Eleanor Boyd (@eleanorjboyd) For what it's worth, I suspect that the first executed subtest determines the checkmark color in the view.
Reacted by Makarov Andrey and Matt HuismanEleanor Boyd (@eleanorjboyd) No! In my first message I wrote:
Exceptions in subtests (except for the first subtest) don't cause the test to be marked as failed.
...
The second subtest has failed but the test is marked as passed on the Testing panel.The main problem is that only the first subtest affects the state of the test as a whole.
This issue was about having the subtests show in the view when using pytest but that is a bug I would want to resolve.
the initial report includes the bug part, that’s probably why this report is marked as both a bug and a feature request. But sure: #25824:
The second subtest has failed but the test is marked as passed on the Testing panel.
Hi Eleanor Boyd (@eleanorjboyd), you mentioned you could provide code pointers if someone wanted to try this, and I would like to take you up on that. I have read your earlier attempt in #24476 and the pytest-dev/pytest-subtests#184 thread, so I know the identifier question was the sticking point before pytest 9 landed.
Here is what I found in the adapter, to check my understanding before I write anything. In
python_files/vscode_pytest/__init__.py,pytest_report_teststatusonly handlesreport.when == "call"and dedups by absolute node id through the module levelcollected_tests_so_farset. A pytest 9 subtest report is a call phase report carrying the parent's node id, so the first SUBPASS or SUBFAIL claims that id and the final parent report is dropped, even though pytest 9 correctly marks the parent as failed with "contains 1 failed subtest". That mechanism explains #25824 as well as this issue.My plan would be two PRs, narrow first:
PR 1, fixes #25824. Detect subtest reports (a pytest 9
SubtestReportcarries acontextattribute) and exclude them from the dedup set so the final parent outcome is published. Small diff, plus a fixture underpython_files/tests/pytestadapterand an expected output entry, version gated so the pytest 8.4 leg of the CI matrix keeps passing.PR 2, this issue. Emit the existing
subtest-failureandsubtest-successoutcomes so the children appear in the tree. The TypeScript side already handles these intestExecutionHandler.ts, andsplitTestNameWithRegexalready parses a parent id followed by a bracketed label, which matches pytest'stest_x [Second subtest]shape.Three questions before I start:
- Is pytest 9 built in subtests enough for a first pass, or do you also want pytest 8 with the pytest-subtests plugin covered? The plugin reports the parent as passed even when a subtest fails, which needs different handling.
- Subtests have no individually callable ids, so the children are display only and their ids have to be synthesized. Is parent id plus the message or kwargs acceptable, including for repeated or unnamed
subtests.test()blocks and for parametrized parents? - You have been reworking this file for performance recently (Improve performance of pytest discovery for large test suites #25974, Enhance test discovery payload handling with compact/expansion on paths in payload #25982, testing: refine compact discovery payload review feedback #26000), and your phrasing about aggregating reports before publishing the final outcome suggests you may prefer buffering until teardown over my simpler skip the dedup approach. Which direction do you want, so I do not conflict with your work?
If this fits, I would be glad to follow up with the teardown failure reporting from #22812 as a third PR. The issue is assigned to you, so please say if you would rather keep it.
Reacted by Makarov Andrey
Type: Bug
Behaviour
I cannot use pytest subtests when pythonTestAdapter is enabled.
Exceptions in subtests (except for the first subtest) don't cause the test to be marked as failed.
Steps to reproduce:
Here's an example:
Test results:
The second subtest has failed but the test is marked as passed on the Testing panel.
Extension version: 2024.20.0
VS Code version: Code 1.95.3 (f1a4fb101478ce6ec82fe9627c43efbf9e98c813, 2024-11-13T14:50:04.152Z)
OS version: Windows_NT x64 10.0.19045
Modes:
python.languageServersetting: DefaultUser Settings
Installed Extensions
System Info
canvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabled
webnn: disabled_off