Repository navigation
Tap parser fails when test name includes non ASCII characters #45706
Description
Version
v20.0.0-pre
Platform
Darwin Moshes-MBP.localdomain 21.1.0 Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:01 PDT 2021; root:xnu-8019.41.5~1/RELEASE_ARM64_T6000 arm64
Subsystem
test_runner
What steps will reproduce the bug?
// test.mjs
import { it } from 'node:test'
it('أهلا', () => {});run ./node --test test.mjs
How often does it reproduce? Is there a required condition?
always
What is the expected behavior?
the same as when running ./node test.mjs without --test:
TAP version 13
# Subtest: أهلا
ok 1 - أهلا
---
duration_ms: 0.6085
...
1..1
# tests 1
# pass 1
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 4.2505
What do you see instead?
TAP version 13
# Subtest: /Users/moshe/repos/node/b.mjs
not ok 1 - /Users/moshe/repos/node/b.mjs
---
duration_ms: 90.51875
failureType: 'uncaughtException'
error: 'Unexpected character: أ at line 1, column 0'
code: 'ERR_TAP_LEXER_ERROR'
stack: |-
Socket.emit (node:events:519:28)
Socket.read (node:net:724:39)
async Promise.all (index 1)
...
1..1
# tests 1
# pass 0
# fail 1
# cancelled 0
# skipped 0
# todo 0
# duration_ms 92.843875
Additional information
according to TAP spec we should support any text:
Description Any text after the test number but before a # is the description of the test point.
Descriptions should not begin with a digit so that they are not confused with the test point number. The harness may do whatever it wants with the description.
the issue seems to be with #isLiteralSymbol
Activity
MoLow commentedon Dec 1, 2022
CC @manekinekko @nodejs/test_runner
[-]Tap parser fails parsing when test name includes non english charactesr[/-][+]Tap parser fails when test name includes non english characters[/+]manekinekko commentedon Dec 1, 2022
Good catch! Will prioritize this and send a fix 👍
[-]Tap parser fails when test name includes non english characters[/-][+]Tap parser fails when test name includes non ASCII characters[/+]pulkit-30 commentedon Dec 2, 2022
Hey @manekinekko @MoLow, can I work on this issue?
MoLow commentedon Dec 2, 2022
@pulkit-30 please do!
manekinekko commentedon Dec 2, 2022
@pulkit-30 of course 👍
murakami commentedon Dec 14, 2022
Thank you for considering the fix.
v19.2.0 I'm very happy because I got an error that Japanese is included in the source.
jammi commentedon Feb 3, 2023
I still have this issue in the newest node at this point (v19.6.0).
Emojis and such are also common characters and not only in the test descriptors but also from code output logging to stdout or stderr, which the tap parser should simply regard as any code output. Otherwise, this would lead to changing the code in order to pass the test suite, which would be silly, especially if there's nothing wrong with anything but the test reporter suite, such as is the case here.
manekinekko commentedon Feb 3, 2023
@jammi can you share a sample of output you have in your code? (please clear any sensitive data first)
MoLow commentedon Feb 4, 2023
@jammi this issue is still open, a fix was not shipped on v19.6.0.
@mertcanaltin any updates regarding #45736?
mertcanaltin commentedon Feb 6, 2023
i will do an update
0618 commentedon Nov 10, 2023
I'm still seeing the issue on v19.9.0
manekinekko commentedon Nov 10, 2023
@0618 would you mind sharing a screenshot or logs?
bnoordhuis commentedon Nov 10, 2023
@0618 opened #50646 but note that the bug is already fixed, just not in node 19 (which isn't supported any longer.)