Repository navigation
Investigate non-ASCII build file path on POSIX #14336
Description
Activity
Stack trace:
➜ 弄得 git:(master) ❯ make ~/wrk/com/弄得 /Library/Developer/CommandLineTools/usr/bin/make -C out BUILDTYPE=Release V=1 LD_LIBRARY_PATH=/Users/gib/wrk/com/弄得/out/Release/lib.host:/Users/gib/wrk/com/弄得/out/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../.; mkdir -p /Users/gib/wrk/com/弄得/out/Release/obj/gen; python tools/js2c.py "/Users/gib/wrk/com/弄得/out/Release/obj/gen/node_javascript.cc" lib/internal/bootstrap_node.js lib/async_hooks.js lib/assert.js lib/buffer.js lib/child_process.js lib/console.js lib/constants.js lib/crypto.js lib/cluster.js lib/dgram.js lib/dns.js lib/domain.js lib/events.js lib/fs.js lib/http.js lib/_http_agent.js lib/_http_client.js lib/_http_common.js lib/_http_incoming.js lib/_http_outgoing.js lib/_http_server.js lib/https.js lib/inspector.js lib/module.js lib/net.js lib/os.js lib/path.js lib/process.js lib/punycode.js lib/querystring.js lib/readline.js lib/repl.js lib/stream.js lib/_stream_readable.js lib/_stream_writable.js lib/_stream_duplex.js lib/_stream_transform.js lib/_stream_passthrough.js lib/_stream_wrap.js lib/string_decoder.js lib/sys.js lib/timers.js lib/tls.js lib/_tls_common.js lib/_tls_legacy.js lib/_tls_wrap.js lib/tty.js lib/url.js lib/util.js lib/v8.js lib/vm.js lib/zlib.js lib/internal/buffer.js lib/internal/child_process.js lib/internal/cluster/child.js lib/internal/cluster/master.js lib/internal/cluster/round_robin_handle.js lib/internal/cluster/shared_handle.js lib/internal/cluster/utils.js lib/internal/cluster/worker.js lib/internal/errors.js lib/internal/freelist.js lib/internal/fs.js lib/internal/http.js lib/internal/linkedlist.js lib/internal/net.js lib/internal/module.js lib/internal/process/next_tick.js lib/internal/process/promises.js lib/internal/process/stdio.js lib/internal/process/warning.js lib/internal/process.js lib/internal/querystring.js lib/internal/process/write-coverage.js lib/internal/readline.js lib/internal/repl.js lib/internal/socket_list.js lib/internal/test/unicode.js lib/internal/url.js lib/internal/util.js lib/internal/v8_prof_polyfill.js lib/internal/v8_prof_processor.js lib/internal/streams/lazy_transform.js lib/internal/streams/BufferList.js lib/internal/streams/legacy.js lib/internal/streams/destroy.js deps/v8/tools/splaytree.js deps/v8/tools/codemap.js deps/v8/tools/consarray.js deps/v8/tools/csvparser.js deps/v8/tools/profile.js deps/v8/tools/profile_view.js deps/v8/tools/logreader.js deps/v8/tools/tickprocessor.js deps/v8/tools/SourceMap.js deps/v8/tools/tickprocessor-driver.js deps/node-inspect/lib/_inspect.js deps/node-inspect/lib/internal/inspect_client.js deps/node-inspect/lib/internal/inspect_repl.js ./config.gypi src/nolttng_macros.py src/perfctr_macros.py touch /Users/gib/wrk/com/弄得/out/Release/obj.host/node_js2c.stamp LD_LIBRARY_PATH=/Users/gib/wrk/com/弄得/out/Release/lib.host:/Users/gib/wrk/com/弄得/out/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../deps/v8/src/inspector; mkdir -p /Users/gib/wrk/com/弄得/out/Release/obj/gen/src; python ../../third_party/inspector_protocol/CheckProtocolCompatibility.py --stamp "/Users/gib/wrk/com/弄得/out/Release/obj/gen/src/js_protocol.stamp" js_protocol.json touch /Users/gib/wrk/com/弄得/out/Release/obj.target/deps/v8/src/inspector/protocol_compatibility.stamp touch cc205296965aac8499af8078773aa6a0978c1a60.intermediate LD_LIBRARY_PATH=/Users/gib/wrk/com/弄得/out/Release/lib.host:/Users/gib/wrk/com/弄得/out/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../deps/v8/src/inspector; mkdir -p /Users/gib/wrk/com/弄得/out/Release/obj/gen/src/inspector/protocol /Users/gib/wrk/com/弄得/out/Release/obj/gen/include/inspector; python ../../third_party/inspector_protocol/CodeGenerator.py --jinja_dir ../../third_party --output_base "/Users/gib/wrk/com/弄得/out/Release/obj/gen/src/inspector" --config inspector_protocol_config.json Failed to parse config file: 'ascii' codec can't decode byte 0xe5 in position 19: ordinal not in range(128) make[1]: *** [cc205296965aac8499af8078773aa6a0978c1a60.intermediate] Error 1 rm cc205296965aac8499af8078773aa6a0978c1a60.intermediate make: *** [node] Error 2cc @nodejs/v8-inspector
I can reproduce this on Linux as well. This is because Python 2's
sys.argvis an array of values of typestr, which is basically an array of 8-bit values, instead ofunicode. The file system functions see astrvalue and interpret it as ASCII, which fails when in Unixsys.argvis in fact encoded with UTF-8.nodejs/node-gyp#1203 might solve this
(trying to upstream https://chromium-review.googlesource.com/c/509632/)@refack It's unrelated.
Hmmmm it's
makenotconfigure... so maybe #12786 (comment)Editing title to say POSIX since this is on Linux as well as macOS.
- changed the title
[-]Investigate non-ASCII build file path under macOS[/-][+]Investigate non-ASCII build file path on POSIX[/+]on Jul 24, 2017 I was able to reproduce this and am looking into it.
- added a commit that references this issue
on Oct 6, 2017 2 remaining items
- added a commit that references this issue
on Dec 12, 2017 - added a commit that references this issue
on Jan 18, 2018 - added 2 commits that reference this issue
on Jan 30, 2018 - added 2 commits that reference this issue
on Feb 27, 2018 - added a commit that references this issue
on May 8, 2018 I will close this as it seems to be resolved (this seems to be the relevant comment #16735 (comment)). I've also built it locally with this path
node/弄得/nodejust fine (Arch Linux).@lundibundi were you able to run
make testin that path?@refack I've done a full
./configure && make -j4 testcommand successfully. Though I'm on Arch and it has python3 by default. I'll now try to do that in python2 env as I forgot about that.Upd: I've successfully built it via
./configure && make -j4 testindev/node/弄得folder using python2 env.
At Code & Learn Shanghai we encountered some issues with non-ASCII (specifically, Chinese) file paths when building. Don't have the Python stack trace, but could someone look into that?