Repository navigation
Jupyter mixes stdout and stderr output in non-deterministic order #173
Description
Description
A dummy progress bar would be created after finishing each outer iteration, looking like below. The code to reproduce the issue is attached. It works fine without print though. Is there any way to avoid this behavior?
100% (100 of 100) |######################| Elapsed Time: 0:00:01 Time: 0:00:01
16% (16 of 100) |### | Elapsed Time: 0:00:00 ETA: 0:00:01
0
100% (100 of 100) |######################| Elapsed Time: 0:00:01 Time: 0:00:01
16% (16 of 100) |### | Elapsed Time: 0:00:00 ETA: 0:00:00
1
100% (100 of 100) |######################| Elapsed Time: 0:00:01 Time: 0:00:01
2Code
import time
import progressbar
for i in range(3):
for j in progressbar.progressbar(range(0, 100)):
time.sleep(0.01)
print(i)Versions
- Python version: 3.6
- Python distribution/environment: Anaconda
- Operating System: macOS 10.13
- Package version: 3.37.1
Activity
wolph commentedon Oct 9, 2018
I cannot reproduce it in either the normal Python console or Anaconda. Can you try in a regular Python console just to make sure it's not related to Anaconda?
Guzzii commentedon Oct 9, 2018
Hi @wolph. Sorry for the confusion. You are right. It works in python/ipython console. It only seems to have issue in Jupiter notebook.
aaronjohnson92 commentedon Oct 14, 2018
I reproduced this on python 3.7 in windows cmd, as well as on arch linux.
wolph commentedon Oct 17, 2018
It seems like a bug in Jupyter outside of my control.
When running this code:
The result is:
That means that the stdout output is being buffered and printed at the wrong time. This is very similar to a bug in the Jetbrains editors (Pycharm specifically, but they all have it): #115
The good news is, as opposed to the bug in Pycharm, this bug can easily be circumvented. Simply add a
sys.stdout.flush()between theprint()and the start of the next bar and you're done :)Guzzii commentedon Oct 17, 2018
Thanks for looking into the issue @wolph . The trick works for me. It is a lot cleaner now
[-]Dummy progress bar in nested for loop with print statement[/-][+]Jupyter mixes stdout and stderr output in non-deterministic order[/+]HDembinski commentedon Apr 14, 2020
@wolph I just stumbled over the same issue. Why can't
progressbar.progressbarcallstdout.flushbefore it is starting to print, to avoid this issue?wolph commentedon Apr 14, 2020
@HDembinski when stdout redirection is enabled it would do at. If stdout redirection is not enabled the progressbar assumes that it's under your control and won't touch it.
Instead of manually flushing that might be a better solution :)
HDembinski commentedon Apr 14, 2020
Sorry, I don't follow what you mean.
I am generally very happy with progressbar, it is well designed and I can see that you care about the details and that things "just work". Perhaps as a workaround for the Jupyter notebook bug progressbar could check sys.stdout, which is an instance of
ipykernel.iostream.OutStreamin a notebook, and in that case flush automatically?wolph commentedon Apr 14, 2020
I was replying from my phone so I couldn't post an example. I mean the output redirection: https://github.com/WoLpH/python-progressbar#combining-progressbars-with-print-output
Although I suppose detecting for that specific stream would be an option too, I'm not entirely sure which implementation would be best here. Automatically enabling the stdout redirection as mentioned above would be the easiest and probably works well in all cases but I'm a bit worried it can break things for people that also do iostream detection.
At the very least I'll need to make sure people can still disable the behaviour just in case they're somehow dependent on it. Wouldn't be the first time ;)
XKCD springs to mind: https://xkcd.com/1172/
HDembinski commentedon Apr 17, 2020
Right, as a fellow maintainer I know. Thanks for reopening.
stale commentedon Jun 17, 2020
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.