Repository navigation
Auto-fill missing leading 0s in URL #2420
Description
I was going to add pep-* autolinking to the python/cpython repo, but I realized that the URL requires the right amount of leading 0 in order to work. For example:
https://peps.python.org/pep-0008/ -- works fine
https://peps.python.org/pep-8/ -- fails
People rarely mention the leading 0s when they write e.g. PEP-8 in a message, and the autolinker doesn't offer any formatting option. Because of this, clicking the link results in a 404.
Would it be possible to create a script that, in case of 404, tries to extract the pep number and redirects to f'https://peps.python.org/pep-{num:04d}/'?
If it's handled on this end, it will have the advantage that every repo will be able to autolink PEPs, and it will also work for people editing the URL manually (something I happen to do from time to time).
Activity
AA-Turner commentedon Mar 12, 2022
I have a more general solution I'd like to try, but won't be immediate (timeframe of weeks not days). What's the urgency on this?
A
ezio-melotti commentedon Mar 12, 2022
The migration to GitHub Issues should happen at the end of the months, but PEP autolinking is not high priority -- the autolinking could be disabled until a fix is in place.
Just out of curiosity, what kind of solution are you thinking about?
AA-Turner commentedon Mar 12, 2022
Two approaches to try -- the first would create a dummy file for each PEP with the zeros stripped (e.g.
8.html) with per file redirects, the second similar to your suggestion of JS in the 404 page.The entire PEPs website doesn't require JS at all though (except for a toggle button for colour schemes), so I want to avoid the second approach.
A
ezio-melotti commentedon Mar 12, 2022
I can see three possible ways of using the dummy files:
The downside of these approaches is that they will create a lot of dummy files, and it will still fail with e.g. PEP-008 or similar.
If you have access to the webserver, you could also configure HTTP redirects or use URL rewrite, but I'm not sure if this is possible with the current setup.
I appreciate the intent of keeping the repo as JS-free as possible, but I think having some JS in the 404 page is ok: if people have JS disabled and click on a broken link they will meet the same 404 page they always met; if they have JS enabled they will get redirected.
Adding a link to the PEP index to the 404 page will also help people without JS to reach the index and from there it's easy enough to
ctrl+fthe PEP number.hugovk commentedon Mar 12, 2022
Rather than running JavaScript client-side (not everyone has JS enabled, it's relatively slow, more code to maintain), how about Nginx server redirect rules (similar to python/pythondotorg#2006) that will update the URL before even hitting the GitHub Pages/RTD website?
Something along the lines of:
\/pep-(\d)$->/pep-000$1e.g./pep-8->/pep-0008\/pep-(\d\d)$->/pep-00$1e.g./pep-12->/pep-0012\/pep-(\d\d\d)$->/pep-0$1e.g./pep-101->/pep-0101(See also https://stackoverflow.com/a/27504007/724176)
Ping also @ewdurbin for comment :)
AA-Turner commentedon Mar 12, 2022
I'll respond to @ezio-melotti's point from #2421 (comment) here too.
My general concern is that whatever we support we will have to support forever, as links will start to propogate with non-canonical forms. Allowing
peps.python.org/\d+to me seems reasonable -- my preference would actually to be stricter and not allow existing zero-padding.The original rationale was that barely anybody writes the leading zeros, so for e.g.
PEP-100there'd still need to be a redirect. (I think we can safely assume those coming from GitHub have JS enabled).For me with awful internet, it still is fairly fast -- I put the redirect script at the top of
<head>so that it would be processed first.I don't think
peps.python.orgis fronted with a CDN currently, so this isn't an option sadly.A
ezio-melotti commentedon Mar 12, 2022
If that's an option, I think it would be ok. Let's see what @ewdurbin says about that :)
The concern is understandable, but wouldn't this introduce a new type of URL (without the
pep-prefix) that we will have to support? If we stick topeps.python.org/pep-<num>/where<num>might or might not have leading0, we will still have one canonical form. IMHO supporting PEP numbers without leading0s is worth the extra effort, since as you also mentioned, barely anybody adds them. The fact that the0s can be added in different ways (e.g. JS and URL rewrite) means that it shouldn't be difficult to support this even if we move to a different environment.CAM-Gerlach commentedon Mar 12, 2022
This originally came up in #2255 .
To note, the canonical way of referencing PEPs in prose text, as defined by PEP 1 and PEP 12, is with no leading zeros and a space instead of a dash. As GitHub apparently does not allow spaces (or requires a dash) in references, it was decided there that requiring the leading zeros too was acceptable, since it made more clear that the number was a GitHub autolink reference rather than a canonical PEP name, to avoid confusing others about the canonical way of referring to PEPs themselves. I typically do so like this: As in PEP 8 (PEP-0008).
That said, if we do add a new URL form, which the non-leading-zero form already is, I would support the shorter and more distinct
peps.python.org/N. Being more clearly distinct from the current form is actually an advantage, as it is more obvious that it is a distinct form, and also ensures unique URLs for all PEPs rather than just <4 digit ones, while avoiding two similar-looking sets of links to the same PEP.It could be done all in HTML without either JavaScript or server-side redirect rules by using
<meta http-equiv="refresh" content="0; url={redirect_url}">combined with auto-generating the redirect pages via a build script. We use such a script that I wrote in production for the Spyder Docs. It could be used here as a post-build step in the CI/makefile, as we do, with only slight modifications (since almost everything is done via CLI arguments rather than hardcoded).AA-Turner commentedon Mar 12, 2022
#2421 now takes the
http-equivapproach (it was simpler than I thought to integrate well in the build process).A
hugovk commentedon Mar 13, 2022
Ah right, peps.python.org goes straight to GitHub Pages (https://hostingchecker.com):
And not into the same place as python.org:
So these server redirects aren't possible.
ewdurbin commentedon Mar 14, 2022
Depending on what decision is made as far as what the best course of action is, we can move peps.python.org behind Fastly and support redirects.
hugovk commentedon Mar 14, 2022
Nice, that sounds best to me, we ran into some problems with #2421.
We'd actually want something like:
\/(\d)$->/pep-000$1/e.g./8->/pep-0008/\/(\d\d)$->/pep-00$1/e.g./12->/pep-0012/\/(\d\d\d)$->/pep-0$1/e.g./101->/pep-0101/\/(\d\d\d\d)$->/pep-$1/e.g./3156->/pep-3156/So
https://peps.python.org/Ngoes to a zero-paddedhttps://peps.python.org/pep-NNNN@CAM-Gerlach @ezio-melotti Is this pattern right? Shall we move peps.python.org behind Fastly and use redirects?
CAM-Gerlach commentedon Mar 14, 2022
That's what @AA-Turner preferred and I thought it sounded reasonable, since it is shorter and more distinct, but I'm not sure @ezio-melotti agreed.
The main problem with #2421 was that
https://peps.python.org/404conflicted with the 404 page, meaning all non-existed pages ended up at PEP 404, which would require either not redirecting for just PEP 404, or reverting to thepep-Nscheme to disambiguate. @ewdurbin does the approach you mentioned allow us to easily avoid that problem?6 remaining items
CAM-Gerlach commentedon Mar 31, 2022
@brettcannon Indeed, that's exactly what we're proposing to do, and AFAIK what @ewdurbin agreed to implement :)
ezio-melotti commentedon Apr 7, 2022
@ewdurbin, what's the status on this?
ewdurbin commentedon Apr 7, 2022
Ironically it's been deprioritized by the hiring process for another Infrastructure team member. I should have time to sit down with this and #2493 tomorrow.
ezio-melotti commentedon Apr 7, 2022
Thanks for the update! The migration to GitHub Issues is scheduled for tomorrow, so the timing is perfect :)
ewdurbin commentedon Apr 8, 2022
Lost time to work on this... due to the GitHub issues migration... perhaps this weekend but likely next week.
ewdurbin commentedon Apr 14, 2022
peps.python.org deployed to Fastly including these redirects.
They are configured as 301s for now.
Overview of the implementation:
https://fiddle.fastlydemo.net/fiddle/cbc02328
ewdurbin commentedon Apr 14, 2022
Please let me know if this is satisfies the requirements by closing this issue :) I'll update to permanent redirects then.
hugovk commentedon Apr 14, 2022
Thanks!
We want the request so we have only digits after
https://peps.python.org/and then the result gets zero padded (width 4) and prefixed withpep-Some examples:
https://peps.python.org/8https://peps.python.org/pep-0008/https://peps.python.org/8/https://peps.python.org/pep-0008/https://peps.python.org/12https://peps.python.org/pep-0012/https://peps.python.org/12/https://peps.python.org/pep-0012/https://peps.python.org/101https://peps.python.org/pep-0101/https://peps.python.org/101/https://peps.python.org/pep-0101/https://peps.python.org/3156https://peps.python.org/pep-3156/https://peps.python.org/3156/https://peps.python.org/pep-3156/We didn't discuss having leading zeros in the request, but I guess these are fine too:
https://peps.python.org/08https://peps.python.org/pep-0008/https://peps.python.org/08/https://peps.python.org/pep-0008/https://peps.python.org/008https://peps.python.org/pep-0008/https://peps.python.org/008/https://peps.python.org/pep-0008/https://peps.python.org/0008https://peps.python.org/pep-0008/https://peps.python.org/0008/https://peps.python.org/pep-0008/ewdurbin commentedon Apr 15, 2022
d'oh. Ok :) https://fiddle.fastlydemo.net/fiddle/71cc868a is the sketch of the service now. changes are active and cover all above scenarios.
ewdurbin commentedon Apr 15, 2022
Opened #2527 to make the fastly service transparent... even if terraform and fastly might be a little obscure 😅
ewdurbin commentedon Apr 15, 2022
I'll close this, as requirements in #2420 (comment) are all addressed.
hugovk commentedon Apr 15, 2022
Thank you, looks good!
Quick demo, these all redirect to their canonical forms:
https://peps.python.org/8
https://peps.python.org/8/
https://peps.python.org/12
https://peps.python.org/12/
https://peps.python.org/101
https://peps.python.org/101/
https://peps.python.org/3156
https://peps.python.org/3156/
https://peps.python.org/08
https://peps.python.org/08/
https://peps.python.org/008
https://peps.python.org/008/
https://peps.python.org/0008
https://peps.python.org/0008/
ezio-melotti commentedon Apr 15, 2022
Thanks for fixing this!
I now updated the autolinking rules of both this repo and
python/cpythonto:PEP-*→https://peps.python.org/*Here are some tests:
lowercase:
pep-8
pep-08
pep-008
pep-0008
uppercase:
PEP-8
PEP-08
PEP-008
PEP-0008
invalid id (404):
pep-123
pep-0123
PEP-123
PEP-0123
Note that old autolinks might take 1 day or so to update, but these should use the new autolinking rule directly.
CAM-Gerlach commentedon Apr 15, 2022
In some sense having the autolink references require the zeros made clear that they weren't actually the correct canonical names for/references to PEPs (even though, admittedly, I was the first to complain about the requirement), but I suppose that's a fairly small (and somewhat pedantic) price to pay for the substantially increased convenience. Thanks @ewdurbin !