Version
v20.13.1
Platform
Darwin, but reproduces on Ubuntu too
Subsystem
No response
What steps will reproduce the bug?
The following test passes on v20.12.2 but fails on v20.13.1 (and v20.13.0):
// test.js
// node --test
import { test } from 'node:test';
import assert from 'node:assert';
test('outer', async (suite) => {
let foo;
suite.before(async () => {
await new Promise((resolve) => setTimeout(resolve, 1000));
foo = 'set value';
});
await suite.test('test', async () => {
assert.ok(foo);
});
});Of note, the before function is awaited. In v20.12, the before properly blocks execution of tests until fully awaited. In v20.13, tests immediately begin executing, meaning the local variable is not set.
$ node -v
v20.12.2
$ node --test
▶ outer
✔ test (1004.618292ms)
▶ outer (1007.326292ms)
ℹ tests 2
ℹ suites 0
ℹ pass 2
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 1063.150791
$ node -v
v20.13.1
$ node --test
▶ outer
✖ test (6.006375ms)
AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
assert.ok(foo)
at TestContext.<anonymous> (file:///Users/sethvargo/Desktop/foo/test.js:13:12)
at Test.runInAsyncScope (node:async_hooks:206:9)
at Test.run (node:internal/test_runner/test:656:25)
at async TestContext.<anonymous> (file:///Users/sethvargo/Desktop/foo/test.js:12:3)
at async Test.run (node:internal/test_runner/test:657:9)
at async startSubtest (node:internal/test_runner/harness:235:3) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: undefined,
expected: true,
operator: '=='
}
▶ outer (7.11475ms)
ℹ tests 2
ℹ suites 0
ℹ pass 0
ℹ fail 2
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 1049.811625
✖ failing tests:
test at file:/Users/sethvargo/Desktop/foo/test.js:12:15
✖ test (6.006375ms)
AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
assert.ok(foo)
at TestContext.<anonymous> (file:///Users/sethvargo/Desktop/foo/test.js:13:12)
at Test.runInAsyncScope (node:async_hooks:206:9)
at Test.run (node:internal/test_runner/test:656:25)
at async TestContext.<anonymous> (file:///Users/sethvargo/Desktop/foo/test.js:12:3)
at async Test.run (node:internal/test_runner/test:657:9)
at async startSubtest (node:internal/test_runner/harness:235:3) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: undefined,
expected: true,
operator: '=='
}and just for good measure, this does not exist in v22:
$ node -v
v22.2.0
$ node --test
▶ outer
✔ test (1007.872875ms)
▶ outer (1009.431875ms)
ℹ tests 2
ℹ suites 0
ℹ pass 2
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 1069.456334
How often does it reproduce? Is there a required condition?
This happens 100% of the time.
What is the expected behavior? Why is that the expected behavior?
I expected TestRunner API stability between v20.12 and v20.13.
What do you see instead?
Tests that previously passed in v20.12 are failing in v20.13.
Additional information
#51909 looks suspicious, but I'm not an expert here.
Version
v20.13.1
Platform
Darwin, but reproduces on Ubuntu too
Subsystem
No response
What steps will reproduce the bug?
The following test passes on v20.12.2 but fails on v20.13.1 (and v20.13.0):
Of note, the
beforefunction is awaited. In v20.12, thebeforeproperly blocks execution of tests until fully awaited. In v20.13, tests immediately begin executing, meaning the local variable is not set.and just for good measure, this does not exist in v22:
How often does it reproduce? Is there a required condition?
This happens 100% of the time.
What is the expected behavior? Why is that the expected behavior?
I expected TestRunner API stability between v20.12 and v20.13.
What do you see instead?
Tests that previously passed in v20.12 are failing in v20.13.
Additional information
#51909 looks suspicious, but I'm not an expert here.