Repository navigation
child_process;stdio: How does the duplex communication work? #321
Description
Consider this code:
// parent.js
const childProcess = require('child_process')
console.log('parent:spawning')
const child = childProcess.spawn('node', ['child.js'], {
stdio: ['ignore', 'inherit', 'inherit', 'pipe']
})
const ipc = child.stdio[3]
child.on('close', i => console.log('parent:evt:child:close', i))
child.on('error', i => console.log('parent:evt:child:error', i))
child.on('exit', i => console.log('parent:evt:child:exit', i))
ipc.on('data', i => {
console.log('parent:evt:data', i)
ipc.write('number 2')
})
ipc.on('close', i => console.log('parent:evt:close', i))
ipc.on('error', i => console.log('parent:evt:error', i))
ipc.on('finish', i => console.log('parent:evt:finish', i))
ipc.on('end', i => console.log('parent:evt:end', i))
console.log('parent:writing')
ipc.write('test')
setTimeout(() => process.exit(0), 10000)// child.js
const fs = require('fs')
console.log('child:start')
setTimeout(() => {
console.log('child:stop')
process.exit(0)
}, 5000)
console.log('child:opening 3')
const writable = fs.createWriteStream(null, {
fd: 3
})
writable.on('error', i => console.log('child:writable:error', i))
writable.on('close', i => console.log('child:writable:close', i))
writable.on('finish', i => console.log('child:writable:finish', i))
writable.on('drain', i => console.log('child:writable:drain', i))
const readable = fs.createReadStream(null, {
fd: 3
})
readable.on('error', i => console.log('child:readable:error', i))
readable.on('data', i => console.log('child:readable:data', i))
readable.on('end', i => console.log('child:readable:end', i))
readable.on('close', i => console.log('child:readable:close', i))
readable.on('readable', i => console.log('child:readable:readable', i))
console.log('child:writing')
writable.write('recv')The output is more-or-less consistently:
parent:spawning
parent:writing
child:start
child:opening 3
child:writing
parent:evt:data <Buffer 72 65 63 76>
child:readable:data <Buffer 74 65 73 74>
child:readable:data <Buffer 6e 75 6d 62 65 72 20 32>
My question is: If the communication is over one fd, which I am assuming is an anonymous pipe, how can the parent and child know to not "receive" the data they have just now written? How does that whole process work?
I've been researching file descriptors, pipes and sockets, to mostly not much avail, so I'm hoping someone here can explain it to me.
Activity
SEAPUNK commentedon Oct 11, 2016
Quoting Wikipedia:
addaleax commentedon Oct 11, 2016
Try adding
console.log('child:is-socket:fd3', fs.fstatSync(3).isSocket())– Node is actually “cheating” here by setting up a socket, not standard unix pipes, so the file descriptors can be used as full duplexes.SEAPUNK commentedon Oct 11, 2016
Ah, I see -- what kind of socket is it? Does it create a file, or listen to a port?
addaleax commentedon Oct 11, 2016
On POSIX, it’s created using
socketpair():If you’re interested in how things work on Windows, you’ll probably have to look at libuv’s source code (at least I won’t be a great help when it comes to Windows 😄).
bnoordhuis commentedon Oct 11, 2016
On Windows,
DuplicateHandle()can magically make a HANDLE (win32 equivalent of a file descriptor) appear in another process.The IPC channel is only used to tell the other process that it has a new HANDLE, it's not passed as auxiliary data like file descriptors are on UNIX. It's kind of neat, really.
SEAPUNK commentedon Oct 11, 2016
So in closing, I can safely assume that the additional stdios I create when I spawn child processes will be full-duplex streams which will automatically close and clean up on either of the processes shutting down? The reason I ask is because I want to have "worker" processes, and I want to have a binary IPC between the main and worker processes.
addaleax commentedon Oct 11, 2016
I think the answer boils down to “yes”. I am not sure how exactly that relates to how the FDs are created, but using
pipeoripcshould always give you a duplex stream.If you want to make sure that changing certain kind of behaviour is at the very least considered a semver-major change by Node, you can try to write a minimal test case and submit that as a pull request at https://github.com/nodejs/node (or look whether there’s already some test covering the behaviour).
SEAPUNK commentedon Oct 11, 2016
Okay, thank you!