Repository navigation
Fail if an invalid -X option is provided #89608
Description
| BPO | 45445 |
|---|---|
| Nosy | @pablogsal |
| PRs |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2021-10-12.13:39:46.718>
labels = ['type-feature', '3.11']
title = 'Fail if an invalid -X option is provided'
updated_at = <Date 2021-10-15.21:39:44.389>
user = 'https://github.com/pablogsal'bugs.python.org fields:
activity = <Date 2021-10-15.21:39:44.389>
actor = 'pablogsal'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2021-10-12.13:39:46.718>
creator = 'pablogsal'
dependencies = []
files = []
hgrepos = []
issue_num = 45445
keywords = ['patch']
message_count = 4.0
messages = ['403733', '403854', '404003', '404052']
nosy_count = 1.0
nosy_names = ['pablogsal']
pr_nums = ['28823', '28972']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue45445'
versions = ['Python 3.11']
Activity
pablogsal commentedon Oct 12, 2021
We found that using -X ---- can be very confusing and annoying when you make typos, so I think having errors if you pass some unknown stuff will be outweighing any downside of making this more strict.
pablogsal commentedon Oct 13, 2021
New changeset db2b6a2 by Pablo Galindo Salgado in branch 'main':
bpo-45445: Fail if an invalid X-option is provided in the command line (GH-28823)
db2b6a2
pablogsal commentedon Oct 15, 2021
New changeset 79bc5e1 by Pablo Galindo Salgado in branch 'main':
bpo-45445: Remove incorrectly commited test file (GH-28972)
79bc5e1
pablogsal commentedon Oct 15, 2021
I'm opening this again as, this is still not finished as we want to still extend the initialization API to allow a non static string to improve the error message.
In general, the protocol is to ask if there is something else to be done before closing (and giving it some time before closing if nobody answers) :)
kovidgoyal commentedon Jun 23, 2022
Your documentation has stated for YEARS that arbitrary values can be passed via PyConfig.xoptions or -X. 9583cac
Quoting your own documentation:
"It also allows passing arbitrary values and retrieving them through the sys._xoptions dictionary."
There is client software out there relying on this. kovidgoyal/kitty#5223
Please do not randomnly break backwards compatibility, without even a deprecation period, because something "annoys" you.
pablogsal commentedon Jun 23, 2022
@kovidgoyal Thanks for your message.
Please, be kind. There are much better ways to communicate your concerns than stating that we "randomly break backwards compatibility because something annoys us" or saying that "Apparently in Python-land its acceptable behavior to break backward
compatibility with documented interfaces on a whim. Bloody joke" as you did in your workaround commit.We all try to do our best here and although I can understand that you are concerned, there is no reason to not be polite.
This is a bit of a grey area because the
_xoptionsis a private attribute so we need to discuss what to do here.Thanks in advance
pablogsal commentedon Jun 23, 2022
CC: @vstinner
kovidgoyal commentedon Jun 23, 2022
I was simply quoting you, you said you made this change because "We found that using -X ---- can be very confusing and annoying when you make typo". That is really not a good enough reason to break backwards compat.
I apologise for my tone, but you guys need to take backwards compat more seriously.
As for this issue, I dont care personally anymore, I have already worked around it. If you insist on breaking backwards compat then I suggest you add some loud warnings to the changelog. Just think of some poor dude running a script with python -X some_typo in it and that script now failing because python went from 3.10 to 3.11.
vstinner commentedon Jun 23, 2022
We take backward compatibility very seriously. In this case, the incompatible change was done on purpose.
It was too easy to make typos and Python would not notice them. For example,
python3.10 -X fautlhandlerdoes not enable faulthandler and Python 3.10 doesn't say anything about the typo. There are many ways to introduce typos:For other command line options, Python is strict. For example,
python -ofails with a fatal error, whereaspython -Oworks as expected. Same for long options like--check-hash-based-pyc always(instead of--check-hash-based-pycs always).I don't see where the -X command line option is documented as it could be used by applications or 3rd party Python modules. It's documented as: Reserved for various implementation-specific options.
https://docs.python.org/dev/using/cmdline.html#cmdoption-X
IMO the most portable way to pass arbitrary options to Python is to use environment variables. It works on any Python versions and any operating system.
By the way, I added sys.orig_argv to Python 3.10: https://docs.python.org/dev/library/sys.html#sys.orig_argv
I'm not comfortable with your tone of your messages.
If you are affected and you consider that the change is not properly advertized in https://docs.python.org/dev/whatsnew/3.11.html I suggest you proposing a pull request to better document the change. Python is maintained by volunteers, you're are not a customer of a paid service.
kovidgoyal commentedon Jun 23, 2022
zooba commentedon Jun 23, 2022
I agree - this was an unacceptable breaking change to the behaviour of the
-Xoption. It's clearly (and still!) documented as accepting arbitrary values, so we can't change that without a deprecation period.pablogsal commentedon Jun 23, 2022
I am fine reverting the change in 3.11 and adding a deprecation period if we all agree that the API is considered public. I think personally I agree with @zooba and the OP, this change should be reverted (at least in 3.11) and a deprecation warning should be emitted.
zooba commentedon Jun 23, 2022
What about deprecating un-namespaced options (namespacing to be bikeshedded)? So if it doesn't start with *waves hands* then it has to match one of our options, but otherwise it's passed through.
Clearly there are people with uses for this, and I've used it myself to inject arguments that weren't suitable for env variables for whatever reason (typically to avoid it being passed on to subprocesses but without having to control subprocess launches). Offering something to migrate to that isn't OS specific seems the least we can do here.
ericsnowcurrently commentedon Jul 5, 2022
If there's a use case for which people have been using -x then we should definitely provide something to meet that need. However, the status quo isn't viable due to the typo problem. A namespace prefix on options could work.
pablogsal commentedon Jul 5, 2022
For the time being, I am opening a revert
pablogsal commentedon Jul 31, 2022
I have merged the revert as the change was technically backwards incompatible and we are super close to RC1:
#94745
We can discuss bringing it back and how for future versions.
gpshead commentedon Aug 5, 2022
Thanks for reverting this. I don't think making
-X arbitrarythingan error is a good idea. It means when we add a -X setting to a version of Python, be it a feature in a minor release or be it a security thing in a patch release, nobody can use it as a flag until they are guaranteed to never encounter specific interpreter versions that support the new flag value. That devalues the concept of -X. We've added -X things to control security fixes in the past and are likely to in the future.I understand the typo problem. But people should be testing that their flags are doing what they intended in some form or another.
At most I suggest just issuing a warning about an unsupported -X flag name when in PYTHONVERBOSE mode.
vstinner commentedon Sep 2, 2024
Ok, let's close the issue as "won't fix".