Repository navigation
IDLE Shell sidebar. #82084
Description
Activity
This issue proposes to add a shell-specific sidebar to Shell. It follows-up bpo-17535, which added a line-number sidebar to module editors and implements the first phase of bpo-37892, which proposes that the single statement editing area of Shell should be strictly just that.
Looking ahead to this issue, bpo-17535 added both a generic Sidebar class and a specific LineNumber subclass. This issue proposes to add a 3 char wide ShellIO subclass that would mark the beginning of the blocks of text added to the Shell history area. Labels, such as '>>>', 'Out', 'Inp', and 'Err' would be used for the first line of user code input, program stdout, user response to input() (there is only one line), and program stderr (which includes tracebacks). I am not quite sure what to do with debug notices and warnings from the warning modules. Maybe use 'Con', maybe use Dbg and Wrn. I expect to test variations.
As with LineNumber, the font face and size will the the same as in the text. The labels should use the highlight colors of their text block except that '>>>' should continue getting the 'console' colors.
I thing the initial implementation should not response to clicks. After experimentation, we might decide that clicking on a label could select whole blocks, but I would like get the essential stuff right first.
Zero commented
on Oct 31, 2019 ZeromannequinMannequinThe documentation for sys.ps1 and sys.ps2 states that they "are only defined if the interpreter is in interactive mode." Since the IDLE shell is meant to be interactive (and to reduce the differences between the shell and running Python directly), would it be helpful if ps1 and ps2 were defined when running IDLE? The shell could then honor their values.
If such a direction was explored, one issue may be that the sidebar could not simply be 3 char wide. The documentation also states that non-strings are evaluated each time they are needed by the interpreter. This allows for such interesting possibilities as shown with the code below but may not be desired functionality for the IDLE shell window.
import sys from datetime import datetime class TimePrompt: def __init__(self, prefix, suffix): self.prefix, self.suffix = prefix, suffix def __str__(self): return f'{self.prefix}{datetime.now()}{self.suffix}'
sys.ps1, sys.ps2 = TimePrompt('', ' >>> '), TimePrompt('', ' ... ')Will the current interactive shell continue to be available?
Stehpen, I'm not sure what your use-case is... To me it seems that sys.ps1 (and ps2?) aren't quite relevant any more when using a sidebar, since there are no longer inline prompts.
Raymond, with the current PR, this replaces the inline prompts. Making it toggle-able could be very awkward. Is this a deal-breaker in your opinion?
Zero commented
on Oct 31, 2019 ZeromannequinMannequinMaybe my impression has been false this whole time, but the Python interactive interpreter seems to be very similar to the IDLE shell window. My question is, "Why not make them even more so?" Having IDLE react to sys.ps1 and sys.ps2 opens up the shell to similar use cases as having them defined in the interactive interpreter. My suggestion is not to have them added as text as is usual in a terminal window but to print the values of ps1 and ps2 in the proposed ShellIO subclass. That would make label customization possible in a way that is already documented and standard.
Stephen, perhaps what you're suggesting is out of context for this issue? This issue is about removing prompts from the shell window's text widget, replacing them with a separate sidebar showing where prompts and user input were. This will make IDLE's shell less similar to the basic command line shell, not more similar.
Zero commented
on Nov 1, 2019 ZeromannequinMannequinZero: "not to have them added as text as is usual in a terminal window"
taleinat: "removing prompts from the shell window's text widget"Zero: "print the values of ps1 and ps2 in the proposed ShellIO subclass"
taleinat: "separate sidebar showing where prompts and user input were"We appear to be in agreement.
terry.reedy: "Labels, such as '>>>', 'Out', 'Inp', and 'Err' would be used"
Zero: "Having IDLE react to sys.ps1 and sys.ps2"My suggestion is to take those labels terry.reedy talks about from the values of ps1 and ps2 since they are already documented and standard for "the interpreter ... in interactive mode." If psX needs to introduced for other prompts that may be needed ("Maybe use 'Con', maybe use Dbg and Wrn."), it would provide a sensible way to customize those labels as well.
After the PR, can someone easily reproduce the interactive prompt sessions in the tutorial and maindocs. Will it look the same? Would they be able to build doctest examples in the IDLE shell?
This is marked for 3.7 through 3.9. Do you intend to backport this PR?
After the PR, can someone easily reproduce the interactive prompt sessions in the tutorial and maindocs.
Yes, except with prompts (both >>> and ...) in the sidebar rather than inline.
Will it look the same?
No, see above.
Would they be able to build doctest examples in the IDLE shell?
That's a good thought. It should be rather simple to add a context menu option to "copy for doctest" which would add the prompts to the copied text.
(By the way, how popular are doctests these days? My impression is that they've completely fallen out of style, but I'm sure that could vary between different areas and industries.)
Stephen (Zero), thanks for the clarification.
FWIW I'm -1 on the ability to customize the prompts in the sidebar. It would complicate the implementation while reducing the straightforwardness and simplicity of IDLE.
Also, anyone who *really* wants to customize them could just change the hard-coded values in the code in their own copy of IDLE. I don't think it's worth the effort to make it more easily configurable.
29 remaining items
Is it possible to add a Settings option under the General tab to switch between the sidebar mode and traditional mode?
As described above in msg392163, the other fix is to put the prompt on a line by itself. I hoped to get a switch between fixes into b1 but ran out of time and energy. I intend to have in in the next, along with other improvements.
Is it possible to add a Settings option under the General tab to switch between the sidebar mode and traditional mode?
Raymond, that should be possible, and possibly rather easily. I'll take a look at adding this option.
Upon reading Terry's previous comments here more carefully, it's clear that he will be adding an option to switch between this mode and an alternate mode with prompts on separate lines in the main text widget. The latter will be much more similar to the traditional mode.
Rather than request a specific solution, I'll state what new problems need to be solved.
For teaching purposes in live demos, it is essential to have a clear visual distinction between the inputs and outputs:
>>> beatles = ['john', 'paul', 'ringo', 'george'] >>> [name.capitalize() for name in beatles] ['John', 'Paul', 'Ringo', 'George'] >>> [name for name in beatles if 'n' in name] ['john', 'ringo']This doesn't work nearly as well:
beatles = ['john', 'paul', 'ringo', 'george'] [name.capitalize() for name in beatles] ['John', 'Paul', 'Ringo', 'George'] [name for name in beatles if 'n' in name] ['john', 'ringo']
With the sidebar, the ps1 prompt helps a little, but gray separator bar visually runs the input and output text together and even causes consecutive inputs to visually be merged. The strongest separator, the unindent, has been lost.
>>>|beatles = ['john', 'paul', 'ringo', 'george'] |[name.capitalize() for name in beatles] >>>|['John', 'Paul', 'Ringo', 'George'] |[name for name in beatles if 'n' in name] >>>|['john', 'ringo']I believe that if you consult a trained graphic designer (i.e. one who can name the 7 elements of graphic design and describe how they are used), they will confirm that the new display is problematic. To make data groups (inputs and outputs) visually distinct, we can change alignment, change vertical spacing, remove strong vertical lines, change color, etc.). But if you align the text and run a vertical line separating it from the PS1 prompt, then the distinction is blurred. At least this is what I was taught in graphic design courses.
Almost the entire purpose of graphic design in an IDE is to help the eye either group together like pieces of information or to help the eye create distinctions between dissimilar things. We give elements the same color if they are related and different colors if unrelated. We bring like information together by placing it close together or by giving it the same alignment. We push things apart and disconnect them by drawing a separator line (in the case at hand, the ps1 prompts are clipped-off in a separate window from the text they were intended to annotate).
If we have to keep the sidebar, some vertical separation would help (as it does in ipython):
>>>|beatles = ['john', 'paul', 'ringo', 'george'] |[name.capitalize() for name in beatles] | >>>|['John', 'Paul', 'Ringo', 'George'] |[name for name in beatles if 'n' in name] | >>>|['john', 'ringo'] |This added spacing would help separate consecutive statements but would cost eating up valuable vertical space and would not help with visually distinguishing the input from the output.
You could take out the vertical line and get an improvement:
>>> beatles = ['john', 'paul', 'ringo', 'george'] [name.capitalize() for name in beatles] >>> ['John', 'Paul', 'Ringo', 'George'] [name for name in beatles if 'n' in name] >>> ['john', 'ringo'] This is better, but could be improved by unindenting to distinguish the inputs and outputs, which just about takes us back to where we started from (not just in IDLE, but what you see in books, presentations, and blog posts): >>> beatles = ['john', 'paul', 'ringo', 'george'] >>> [name.capitalize() for name in beatles] ['John', 'Paul', 'Ringo', 'George'] >>> [name for name in beatles if 'n' in name] ['john', 'ringo']Another desirable feature is the ability to cut and paste snippets into docstrings. This is essential not just for doctest, but even for untested examples in docstrings.
Those examples, need to be recognizable to humans as interactive sessions with outputs clearly distinguished from inputs. When I tried out the new sidebar in prep for a class, it was impossible to include the ps1 prompts in a cut and paste selection. They had to be manually typed back in.Lastly, for live demos in presentations, it is desirable to have a clear screen, free of distracting artifacts. The sidebar is such an artifact -- you would never see such a thing in a book or slide presentation. Also, for live demos, large fonts are necessary which means that screen space is at a premium and the loss of horizontal space matters.
Yes, lets discuss actual problems and various possible solutions.
Raymond's example is incomplete as it omits multiline statements, which is where the real problems lie and the reason a fix was sorely needed. This has been more or less agreed on since at least 2005.
Old:
------------------------------>>> if a: print('okay:' else: if a is None: print('none:') else: print('else:')else:
------------------------------Possible confusion between input and output is a pre-existing issue. For the default themes, Shell User Output has the same background color as Code or Normal Text. The foreground is blue. The lack of contrast is worse on the dark theme. I think using the same background was a mistake, but so far changing the default has been considered off limits. The pre-existing fix for distinguishing output is to customize colors for Shell User Output, as I have done. I have also given Shell User Exceptions more contrast. One can make both as different as one wants.
In 2014, bpo-7676, Raymond said that the indent mess was a "major PITA" and the impossibility of copy-pasting correctly indented code into an editor "has been a continual source of frustration for students in my Python courses as well. I'm looking forward to it being fixed."
At the time, I had no more idea how to fix this than Kurt Kaiser did in the 2000s. But I eventually realized that the problem is entirely due to the first line of multiline statements being indented relative to following code lines, unlike the REPL. The fix (and the only acceptible fix) was and is to stop doing that.
The day before the beta1 release*, Tal and I merged a beta-quality release that we said would need improvement before production release.
------------------------------>>> if a: ... print('okay:' ... else: ... if a is None: ... print('none:') ... else: ... print('else:') ... else:
I am sure that a graphics designer would consider this improved. Unindented code now lines up, and the indents are that same as in any decent python-aware editor. In any case, the above is identical to the REPL except that in the REPL, the output starts at the left, under the prompts.
If one selects the above lines, easily done by clicking and dragging on the sidebar, just as with line numbers, right clicks, and selects 'Copy with prompts' on the context menu, one gets the above *with* output 'else' to the left. IDLE is now as useful for interactive snippets as the REPL. This is mentioned in the IDLE What's New and News items.
I can imagine a possible alternative fix that would put output under the prompts in Shell, not just in the clipboard copy. It would involve a fixed input prompt above the code entry area and side prompts in the history area. This would be a variation on the two-text solution that I discussed on bpo-37892.
As for copying code to paste into an editor: As I believe was mentioned above, we had 'Copy only code' on the context menus, but removed it because it also copied user input in response to user code prompts, which is currently tagged the same as code entered in response to the Shell's prompts. Fixing this will be a high priority issue. When it is, we will list both new option on the Shell menu.
I will soon open another issue for the sidebar colors. I consider the current colors to be placeholders. We set that aside to get the sidebar otherwise ready. Some might prefer a background matching the general shell background so that there is no vertical separation.
- Not by choice. We were delayed 6 months because Azure Pipelines, on Ubuntu and Windows, changes the test environment by grabbing sys.stdout. Shell receives input on sys.stdout. So when test code sent text to shell via sys.stdout, it went to Pipelines instead of Shell, breaking the tests. We should have ignored those failures, but didn't. Once we realized the problem, less that 2 weeks before the beta, we had the test steal sys.stdout back for each test and moved forward.
For teaching purposes in live demos, it is essential to have a clear visual distinction between the inputs and outputs:
>>> beatles = ['john', 'paul', 'ringo', 'george']
>>> [name.capitalize() for name in beatles]
['John', 'Paul', 'Ringo', 'George']
>>> [name for name in beatles if 'n' in name]
['john', 'ringo']Thanks for bringing this up. Lack of clear separation of inputs and outputs could be considered a drawback of putting prompts in a sidebar.
Considering that the prompts *are not part of the code*, I do believe separating them more clearly is a benefit. To support this, consider that in Jupyter notebooks, the prompts are also placed in a graphically-separated area to the left of both input and output "cells".
Also, do note that IDLE colors outputs differently from inputs, which helps differentiate them from inputs very clearly IMO. Therefore, the examples in messages posted here are lacking in this regard. For context, I'm attaching an image with screenshots before and after the change to use a sidebar.
Also, consider that previously, separation of outputs from inputs was not great for multi-line statements, as indentation was inconsistent and more vertical space was used:
>>> if True: print('Foo') else: print('Bar')
Foo
I think that moving prompts into a sidebar is an improvement in this regard:
>>| if True:
...| print('Foo')
...| else:
...| print('Bar')
...|
| FooTaking these three points together, I the sidebar clearly improves the separation of prompts from actual code, and the differentiation of outputs from inputs does not seem much worse, and is arguably overall better.
For a wider context, besides the default REPL, I also consider two other leading REPLs: ipython and Jupyter notebooks. They both also have clear separation of outputs from inputs, though in different ways than the standard REPL.
ipython retains similar behavior to the standard REPL in this regard, replacing the '>>>' prompts with "In [1]:", "In [2]:" etc. It also prefixes outputs of statements with e.g. "Out[2]:", but not outputs written to stdout. And it has additional vertical spacing between commands.
Here is a short example session:
Python 3.8.5 (default, Jan 27 2021, 15:41:15)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.18.1 -- An enhanced Interactive Python. Type '?' for help.In [1]: if True:
...: print('Foo')
...:
FooIn [2]: beatles = ['john', 'paul', 'ringo', 'george']
In [3]: [name.capitalize() for name in beatles]
Out[3]: ['John', 'Paul', 'Ringo', 'George']Jupyter notebooks are a different beast since their interface is browser-based rather than terminal-based, and they take more advantage of non-text graphical elements than IDLE's shell. Inputs are done in "cells", which may contain multiple statements, and each cell its own output. These cells are clearly visually separate from each other. I've attached a screenshot of a similar Python 3 session in a Jupyter notebook for comparison.
Indeed, both the ipython REPL and Jupyter notebooks go further than the default REPL in differentiating outputs from inputs, using different methods.
Another desirable feature is the ability to cut and paste snippets into docstrings. This is essential not just for doctest, but even for untested examples in docstrings.
Those examples, need to be recognizable to humans as interactive sessions with outputs clearly distinguished from inputs. When I tried out the new sidebar in prep for a class, it was impossible to include the ps1 prompts in a cut and paste selection. They had to be manually typed back in.This is no longer the case. In a quick followup PR, which has been merged for some time now, we added a context-menu option "Copy with prompts" which results in code samples directly usable for doctests and code samples.
Lastly, for live demos in presentations, it is desirable to have a clear screen, free of distracting artifacts. The sidebar is such an artifact -- you would never see such a thing in a book or slide presentation. Also, for live demos, large fonts are necessary which means that screen space is at a premium and the loss of horizontal space matters.
I am not sure I agree with the strength with which you state "you would never see such a thing in a book or slide presentation". With Jupyter notebooks being very prevalent these days, samples very often include the non-code visual artifacts native to that format. Arguably, the *prompts* are a distracting visual artifact; the ability to cut them out of screenshots from Jupyter notebooks thanks to the clearer separation of cells and their inputs and outputs is indeed a boon, and we could consider whether we can do something similar for IDLE.
As for lost vertical space, that is indeed at a premium for books and presentations. However, ipython significantly increases the vertical spacing to the left of code inputs and non-stdout outputs compared to the standard Python REPL and IDLE. And the sidebar adds 3.5 characters' width only for outputs, which can be avoided when it's truly crucial using the "Copy with prompts" feature. Overall, considering the benefits, this seems like a relatively minor drawback.
Other bug reports and feature requests that matter to us don't appear to be getting any traction:
I'll start going through those, and see if I can fix some of the bugs on that list.
A few notes:
Need an option to run the install-certificates script
Are you suggesting that we add a way to do this directly from IDLE?
Sometimes the Search Dialog window becomes unresponsive and there is no way to clear it without turning IDLE off.
Is there an open issue about this? Details would be helpful.
Several of these issues have only came-up in the past few years. IDLE used to be more stable than it is now.
This may be true, and at least partially due to a continuous need to fix things broken by various OS and Tcl/Tk changes.
For mac users who have enabled tabs system-wide, opening a new window in IDLE triggers a system log-out (amazing that this is even possible). My understanding is there isn't anything we can do about this, but it is a really bad out-of-the-box user experience seconds after a fresh install of Python.
This is a prime example of the above: macOS changes + Tcl/Tk issues with those mean we are often very limited in our ability to properly address such issues. FWIW we added a warning in the IDLE shell about running with that OS option enabled, see bpo-34864 and PR #54673. I wasn't aware of such crashes; is there an issue about them?
Being able to run pip from within IDLE would be a major win, especially for Windows users who are foreign to the command-line and who often do have Python on the PATH.
I agree! This needs to be properly designed and would be far from trivial to implement and test. If an issue about this exists or would be created, I'd be happy to help move it forward.
I will link to an IDLE shell followup issue later.
Discussions about other IDLE issues should be on pre-existing issues (use the tracker search with Components: IDLE), new issues, or without an issue, on the idle-dev list.
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields: