Repository navigation
Add command-line test filtering to node:test #42984
Description
Activity
@nodejs/test_runner
Are there any test runners that provide a facility like this?
I’d expect to be able to filter tests on the CLI with file globs, and perhaps with a way to grep against test suite names, but this functionality doesn’t seem to have precedent.
Pretty much all test runners do! For example, just looking at the top 5 runners from the State of JS survey:
- Mocha, as mentioned, has
--grep, among other filtering options - Jest has
--testNamePattern - Vitest has
--testNamePatternas well - Playwright has
-gas well as the ability to run tests declared on specific lines - ava has
--match - nose, just to throw in a non-JavaScript test runner allows specifying tests in the form
module:TestCase.test_method
All of these allow filtering to specific test cases to run. The
grepandpatternoptions generally work on a full test or suite name formed from concatenating it with all of its parents. For example,top level test subtest 1.As mentioned, a notable difference is that these all expect side-effect-free parent tests, so I'm not sure a grep-like option would work for
node:test(though maybe I'm wrong.) That's why I suggested selecting by an explicit full path instead.Reacted by Vasyl Boroviak and Feng Yu- Mocha, as mentioned, has
Test name patterns are a feature I’d expect; but “path to the test” and framing this with “only” seems a bit confusing.
I think the expectation of no side effects is a good one; a test that doesn’t do setup/teardown properly is broken.
Test name patterns are a feature I’d expect; but “path to the test” and framing this with “only” seems a bit confusing.
I'm not sure how name patterns would work in the current Node API. Most of the tools mentioned have a differentiation between "suites" and "tests". "suites" are expected to be side-effect free when you call them. This allows tools that run with "grep" to evaluate all suites to enumerate each test, and then match and invoke the test functions for the relevant ones. However,
node:testdoes not have this differentiation.For example say I have some math tests and want to test addition, in Mocha I might write:
suite('math', () => { suite('hard', () => { test('factorization', () => { factorSomeGiantPrimeNumberThatsVerySlowAndTakesAMinute() }) }) suite('simple', () => { test('addition', () => { assert.strictEqual(2 + 2, 4) }) }) })
I could call this by running
mocha -g "addition". In this case, mocha will call the closure provided to thesuites, enumerate the test cases, and then run only theadditiontest.In
node:test, this would instead be written as:test('math', async (t) => { await t.test('hard', async (t) => { await t.test('factorization', () => { factorSomeGiantPrimeNumberThatsVerySlowAndTakesAMinute() }) }) await t.test('simple', async (t) => { await t.test('addition', () => { assert.strictEqual(2 + 2, 4) }) }) })
Say there was now some hypothetical argument where I could run
node --test-grep addition. How would that work? The runner does not differentiate between tests and suites, so it could not enumerate the tests without also running them, since anyt.testmight have nested subtests. From a user's point of view, if I want to only run my "addition" tests, I do not want my very slow "factorization" test to also run and cause me to wait a minute for results instead of milliseconds.That's where my "path to tests" proposal originates -- if the runner knows the path to the tests, it can skip calling (i.e. just
return Promise.resolve()) for tests which are outside of the requested path.Reacted by Vasyl BoroviakThe CLI filtering is the feature all IDEs use to implement "click to run single test" buttons.
Frankly, I never run unit tests from CLI myself. Only via my IDE.
Thus, this feature is the deal beaker for me to use the new node test runner.
- added a commit that references this issue
on Oct 11, 2022 - added a commit that references this issue
on Dec 9, 2022 - added a commit that references this issue
on May 22, 2026
What is the problem this feature will solve?
It's interesting to see
node:testin core, and I wanted to investigate building a VS Code extension for it using our testing API so that users could run it from their UI (e.g. vitest).Currently, running tests can solely be provided by modifying code with
only. This is quite problematic, since there are many UI gestures--in any test UI, not just VS Code--which rely on running specific tests. For example, clicking on a single test to debug, or more advanced actions like re-run failed tests. From my understanding of the docs, these cannot be implemented without requiring changes to code itself.What is the feature you are proposing to solve the problem?
I request some way to narrow tests from the command line. The means of this are negotiable -- mocha has a "-g" option that allows grepping for full test names. However, this is based on having side-effect-free
suites that allow tests to be enumerated and then grepped on, whichnode:testdoes not have.Instead, perhaps an argument like
--test-only-paths="<json>", where<json>is an array of test paths, such as:Not very human-friendly, but easy for machines to deal with.
What alternatives have you considered?
We could alter the source code on disk to pass
onlyparameters totest(). But this is pretty dangerous, if the transformation goes wrong or if the UI crashes before the transformation is rolled back. The closest we can get today is narrowing down to a single test file to run rather than the entire project, but this is still not a good experience.Alternatively, we could use proxyquire or similar to wrap
node:testand inject our own "only" running. This is quite complicated, however, and I don't think it would work with ES modules.