Repository navigation
unable to deal with file names that have invalid encoding #128
Description
There is a file in my user's file system and io.js cannot use any of the fs API on it, because the file name has an invalid encoding. File names are byte arrays; not strings. This is the root of the problem.
How to reproduce (use linux):
$ wget https://s3.amazonaws.com/superjoe/temp/encoding-test.tar.gz
$ tar xvf encoding-test.tar.gz
encoding-test/
encoding-test/test.js
encoding-test/dir/
encoding-test/dir/\377
$ cd encoding-test/
$ cat test.js
var fs = require('fs');
var path = require('path');
var dir = 'dir/';
var list = fs.readdirSync(dir);
var filename = path.join(dir, list[0]);
console.log(filename);
console.log(fs.lstatSync(filename));
$ node test.js
dir/�
fs.js:688
return binding.lstat(pathModule._makeLong(path));
^
Error: ENOENT, no such file or directory 'dir/�'
at Object.fs.lstatSync (fs.js:688:18)
at Object.<anonymous> (/home/andy/Downloads/encoding-test/test.js:7:16)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Function.Module.runMain (module.js:497:10)
at startup (node.js:119:16)
at node.js:906:3
Activity
bnoordhuis commentedon Dec 9, 2014
It's a known issue, see e.g. nodejs/node-v0.x-archive#2387. (There is another, older issue but I can't find it just now.)
There is currently no workaround. Fixing it probably entails rewriting src/node_file.cc and lib/fs.js to accept both buffers and strings (and, perhaps, typed arrays.)
Most platforms treat file paths as byte arrays but Windows and OS X do not; I'm not sure if that would affect node.js but it's something to keep in mind. It's possible that the Windows port of libuv already takes care of all the details but the OS X port most assuredly does not.
At any rate, pull requests are welcome!
piscisaureus commentedon Dec 9, 2014
Libuv for windows assumes that filenames are valid UTF-16 and converts those to and from UTF-8.
However on windows are really uint16 arrays; they are not necessarily valid UTF-16, so a similar issue exists on windows.
chrisdickinson commentedon Dec 16, 2014
If possible I'd like to avoid duplicating issues from the node tracker into the iojs tracker. Closing this in favor of the original issue.
andrewrk commentedon Jan 19, 2015
I don't understand, isn't io.js a fork? Shouldn't it have independent issues? Let's leave the issue open in io.js until it's resolved. I'll see about making a pull request to make src/node_file.cc and lib/fs.js accept buffers in addition to strings.