Repository navigation
How to call a python child process as a server and redirect output in real-time? #103
Description
Activity
- AFAIK you cannot perform redirection when launching a process like this, this is performed by your shell (such as bash). In fact, this is exactly what python-shell does under the hood: it redirects stdin and stdout between Python and Node. So stdout is already redirected to your Node program, but stdout is being buffered by default (which is a good thing most of the times). Output will not exactly be returned at the end of the Python process, it is being buffered and Node will receive a chunk only when the buffer is full, or when the program terminates. Maybe you can try disabling stdout buffering for your Python program using “-u” (see https://stackoverflow.com/questions/107705/disable-output-buffering), but that would kill performance. Most python-shell users control their stdout and can flush explicitly when needed. However, in your case, you might not have control over the output written from bokeh. Personally, I would keep buffering. In real-world situations where you have constant traffic, your buffer will fill normally and your Node program will get near-real-time data.…Sent from my iPhone________________________________ From: chesucr <notifications@github.com> Sent: Thursday, January 18, 2018 8:36:54 AM To: extrabacon/python-shell Cc: Subscribed Subject: [extrabacon/python-shell] How to call a python child process as a server and redirect output in real-time? (#103) I would like to run the bokeh server as a child process with Node. I would like to use this package and redirect the output in real-time, not at the end of the execution. Bokeh does not have a configuration file or argument where I can write the logger file path. So I need to use command redirection operators<https://technet.microsoft.com/en-us/library/bb490982.aspx?f=255&MSPPError=-2147217396> in order to reroute the logger: '>> `${debug_path}` 2>&1' So I have tried some things: process.chdir(`${path}`); // path where I should run bokeh var options = { mode: 'json', pythonPath: `${python_path}`, pythonOptions: [ '-m', 'bokeh', 'serve', ], args: [ '>>', `${debug_path}`, '2>&1', // this is not working, in pythonOptions neither ] }; python_shell.run('', options, function (err, results) { if (err) { logger.error(`${err}`); } // results is an array consisting of messages collected during execution if (typeof(results) !== 'undefined') { logger.info(results); // this is run at the end of the execution if it was an script } }); However I can do it with exec method. But it has some drawbacks, if the node process is killed by forced, the python process become a zombie process. So this is not a good solution: // command is a string with the whole paths and arguments concatenating '>> `${debug_path}` 2>&1' at the end shell = child_process.exec(command, (error, stdout, stderr) => { if (error) { logger.error(`${error}`); } logger.info(`stdout: ${stdout}`); logger.info(`stderr: ${stderr}`); }); Is this possible? I will appreciate any help or piece of advice — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#103>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ABfIVfny05E0xjBsMyHrBIAJDgAi1I9Kks5tL0j1gaJpZM4Ri7mC>.Reacted by Almenon and ChesuCR
I was wondering why my print results were not in real-time. Thanks @extrabacon
@extrabacon thanks for your attention
Yes, I am afraid the bokeh logger is not under my control. Only if I could write directly into a file the problem would be solved.
python_shelltakes the info from thestdoutif I use theprintpython function. But, what is happening with the information printed with the bokeh logger or the python logger by default? It just disappears? Imagine this simple python scriptimport logging import sys logger = logging.getLogger() logger.info('before redirect stream') logging.basicConfig(stream=sys.stdout, level=logging.INFO) logger2 = logging.getLogger() logger2.info('python logger message') print('print content')The
stdoutobject takes the last both messages well. But it is not taking the first one, is it vanished? I am wondering if there is a way to redirect the bokeh logger stream.Well, anyway I think this shouldn´t be a
python_shellissue. So I made a request to change the logging output in the bokeh repository
I would like to run the bokeh server as a child process with Node. I would like to use this package and redirect the output in real-time, not at the end of the execution.
Bokeh does not have a configuration file or argument where I can write the logger file path. So I need to use command redirection operators in order to reroute the logger:
So I have tried some things:
However I can do it with
execmethod. But it has some drawbacks, if the node process is killed by forced, the python process become a zombie process. So this is not a good solution:Is this possible? I will appreciate any help or piece of advice