Repository navigation
add \z as a synonym for \Z in Python REs for standardization #133306
Description
Feature or enhancement
Proposal:
Hello - I’m with the Austin Common Standards Revision Group - the joint technical working group established to develop and maintain the core open systems interfaces that are the POSIX™ 1003.1 (and former 1003.2) standards, ISO/IEC 9945, and the core of the Single UNIX Specification.
We have had a request to unify/rationalize the regex behaviors for “anchor at string beginning” (^ is the closest in POSIX) and “anchor at string end” ($ is the closest in POSIX). A description of this problem in depth can be found here and a table that scopes the varied solutions across varying languages can be found here.
Our working group has come to the conclusion that \A and \z are widely implemented across many ecosystems and are the most “standard” solution to the issue. We are asking if the Python community would consider adding “\z” as a synonym for “\Z” in their regex lexicon.
Has this already been discussed elsewhere?
I have already discussed this feature proposal on Discourse
Activity
david-a-wheeler commentedon May 2, 2025
+1, I'm very supportive!
Currently it's easy to make a mistake using regexes when checking inputs for security; see Correctly Using Regular Expressions. Many platforms interpret
\Zas "optional newline followed by end-of-string" including Java, .NET/C#, Perl, PCRE, PHP (using PCRE), and Ruby. Many developers aren't aware that regex syntax varies between programming languages, and LLMs "borrow" regexes from other languages without necessarily fixing them. If everyone supported\Afor beginning and\zfor the end, we could at least have unified regular expression syntax for this common case (beginning of string .. end of string).Thanks!
collinfunk commentedon May 2, 2025
Hi @serhiy-storchaka, before you spend too much time implementing this I think you should be aware that there is some discussion on the glibc mailing list [1]. Because it is a proposed standardization, it is still subject to change. Paul Eggert, a glibc maintainer, wants to standardize
\`and\'which is already in glibc and other implementations.[1] https://inbox.sourceware.org/libc-alpha/d357ebe7-9aef-41f7-98da-5fa891f8064a@cs.ucla.edu/T/#m6dcdfc0466b896647fce7be5569d8d39af797125
pythongh-133306: Support \z as a synonym for \Z in regular expressions
sethmlarson commentedon May 2, 2025
As the author of a blog post about this footgun, I am in complete support of this proposal! Thank you for opening this issue.
serhiy-storchaka commentedon May 2, 2025
\`and\'is a terrible option.\'usually happens at the end of the regular expression, see what it looks like:r'\`"[^"]+"\''. It is difficult to see where the end of the string. Even if you use double quotes, adjacent'and"are hard to red:r"\`'[^']+'\'".Even if
\`and\'be standardized,\Ais already supported in Python, and\zis supported in many other engines.\Aand\zwill left a preferable syntax for these anchors.mstevenbrown commentedon May 2, 2025
The glibc discussion was also started by our committee, as a part of the rationalization effort.
collinfunk commentedon May 2, 2025
Yes, I experienced that in typing my original message. 😄
Anyways, I don't have super strong opinions. I just wanted to make sure the disagreement was known and considered. Thanks!
gh-133306: Support \z as a synonym for \Z in regular expressions (GH-…
serhiy-storchaka commentedon May 3, 2025
Thank you for opening the discussion and the issue, @mstevenbrown. Thank you for additional link, @@collinfunk
\zhas been added as a synonym for\Z.Some notes:
AFAIK,
\Zis an error inherited from PCRE 0.95 on which the first implementation of theremodule was based. It was fixed in PCRE 2.0, and\zwas added (https://www.pcre.org/original/changelog.txt), but it was stuck in Python for 27 years.Tcl is another programming language that uses
\Zfor end-of-string. Perhaps it has a similar history, if was not borrowed from Python.\Zmay be deprecated and removed in the future, but not before all supported versions of Python support\z. It is less likely that\Zwill be re-introduced with a Perl-compatible semantic (which is equivalent to(?=\n?\z)).\`and\'are very inconvenient for Python.\'followed by the closing quote is hard to read. In a non-raw string literal it should be spelled as\\'or\\\', it is easy to make mistake.\`can be confused with\'or a grit on a screen (especially if it was "typographically enhanced", e.g. https://docutils.sourceforge.io/docs/user/smartquotes.html).\`on GitHub and other Markdown-based forums.\'in a shell script or command line.\`and\'mean literal`and'. Many users escape punctuation characters in regular expressions if they are unsure whether escaping is necessary. This is a breaking change and would require transitional period of emitting FutureWarning.So, even if
\`and\'are standardized and their support is added to Python (after a few painful years),\Aand\zwill remain the preferred ways to use these anchors in Python.pythongh-133306: Use \z instead of \Z in regular expressions in the s…
pythongh-133306: Use \z instead of \Z in fnmatch.translate() and glob…
pythongh-133306: Use \z instead of \Z in regular expressions in the s…
gh-133306: Use \z instead of \Z in fnmatch.translate() and glob.trans…
gh-133306: Use \z instead of \Z in regular expressions in the stdlib (G…
david-a-wheeler commentedon May 5, 2025
Thank you VERY much, everyone!
david-a-wheeler commentedon May 6, 2025
I've submitted a ticket to TCL to add support for
\zin addition to\Zfor end-of-data. It was a 1-line change, with additional changes to update the documentation. Obviously they decide what to do with it, but I thought you might want to know since TCL was mentioned. The specifics are here: https://core.tcl-lang.org/tcl/tktview/fbc56b259e989230e54a4053feeecf7aa765f61dpythongh-133306: Support \z as a synonym for \Z in regular expressions (
pythongh-133306: Use \z instead of \Z in fnmatch.translate() and glob…
pythongh-133306: Use \z instead of \Z in regular expressions in the s…