Repository navigation
Replace bundled pip and setuptools with a downloader in the ensurepip module #80789
Closed as not planned
Description
| BPO | 36608 |
|---|---|
| Nosy | @ericvsmith, @ned-deily, @serhiy-storchaka, @dstufft, @webknjaz, @pradyunsg |
| PRs | ensurepip #12791 |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2019-04-11.22:13:02.045>
labels = ['3.8', 'type-feature', 'library', '3.9']
title = 'Replace bundled pip and setuptools with a downloader in the ensurepip module'
updated_at = <Date 2021-07-05.23:28:08.628>
user = 'https://github.com/webknjaz'bugs.python.org fields:
activity = <Date 2021-07-05.23:28:08.628>
actor = 'ned.deily'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2019-04-11.22:13:02.045>
creator = 'webknjaz'
dependencies = []
files = []
hgrepos = []
issue_num = 36608
keywords = ['patch']
message_count = 6.0
messages = ['339998', '340039', '340060', '340169', '340218', '347836']
nosy_count = 6.0
nosy_names = ['eric.smith', 'ned.deily', 'serhiy.storchaka', 'dstufft', 'webknjaz', 'pradyunsg']
pr_nums = ['12791']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue36608'
versions = ['Python 3.8', 'Python 3.9']
Activity
webknjaz commentedon Apr 11, 2019
Hi,
I've noticed that there's an idea to not pollute Git tree with vendored blobs. In particular,
ensurepipis one of the components doing this.Such a wish was expressed here: https://bugs.python.org/issue35277#msg330098
So I thought I'd take a stab at it...
ericvsmith commentedon Apr 12, 2019
ensurepip does not access the network, by design. We do not want it to start access the network without a lot of discussion.
And if it does access the network, it will need to be able to use alternate URLs. For example: where I deploy Python, it would not have access to the URLs in your PR, but instead would need to specify a different (internal) location. This is the same reason that pip install has --find-links, --no-index, --extra-index-url, etc.
I think this would need a lot of discussion (probably on distutils-sig), and probably a PEP.
ericvsmith commentedon Apr 12, 2019
And I don't mean to sound like a total downer. I just think it's important that we recognize all of the use cases.
Thanks for your work on this.
pradyunsg commentedon Apr 13, 2019
(Not sure how the Roundup handles email replies but I'm hoping this goes to
the right place)
I think it would be better if the downloading got invoked during the
interpreter build process -- to download the wheels and add them to the
final distribution. This lets us remove the wheels from the source
tree/version control and prevents needing to change the PEP for this change.
Functionally, I imagine having all the download logic in some sort of
ensurepip._bootstrap which has all the download logic and that getting
invoked in the build process.
python -m ensurepipnot hitting theinternet is a good invariant to keep.
On Fri, 12 Apr 2019 at 8:54 PM, Eric V. Smith <report@bugs.python.org>
wrote:
serhiy-storchaka commentedon Apr 14, 2019
I proposed to move bundled pip and setuptools to the external repository and download them at build time like Tcl and other dependencies on Windows.
webknjaz commentedon Jul 13, 2019
Thanks for the feedback!
I've changed it a bit to have a separate command for downloading bundles to the source tree. It'd work as in
python -m ensurepip.bundle(needs a better name/CLI args probably).Does it sound better now?
👷Implement build-time pip bundling @ `ensurepip`
👷Implement build-time pip bundling @ `ensurepip`
👷Implement build-time pip bundling @ `ensurepip`
20 remaining items
👷Implement build-time pip bundling @ `ensurepip`
👷Implement build-time pip bundling @ `ensurepip`
👷Implement build-time pip bundling @ `ensurepip`
👷Implement build-time pip bundling @ `ensurepip`
👷Implement build-time pip bundling @ `ensurepip`
webknjaz commentedon Oct 11, 2023
@pradyunsg should this issue get newer labels for
3.10,3.11,3.12and3.13?pradyunsg commentedon Oct 12, 2023
I discussed this at some length at the core developers sprint this week (including some of current RMs).
The main benefit of doing this is that we can avoid committing a ~2 MB wheel1, on a quarterly basis for pip releases, due to a limitation of git + binary blobs. Given that we no longer vendor a setuptools wheel, which evolves at a faster cadence than pip, the size growth rate as well as the rate of change isn't particularly high.
The costs of this are (i) every one who tries to build CPython for the first time and run tests would need to perform this download, (ii) the CPython release process needs to be amended to include a downloaded wheel as part of the release tarball, (iii) it's unclear how/when this downloaded wheels should be deleted/removed and how that interacts with existing
make clean/distcleanetc, and (iv) it introduces an additional point of interaction with PyPI as part of the development/release cycle for CPython.Based on the additional side-effects and work that this would cause, I'm gonna say that this isn't a big deal for now. We can revisit this if there is a strong(er) argument against committing the wheels into the repository to justify the additional side-effects on the development and release processes that it would have.
Footnotes
Or a few more, depending on bugfixes and whether pip's RM decides to place every wheel. ↩
SpecLad commentedon Oct 12, 2023
Would it perhaps be viable to include pip as an (unpacked) sdist in the ensurepip source tree, and build it during the CPython build process? The repo bloat should be much smaller that way.
(You'd also need to include pip's build dependencies, but those don't need to be updated frequently.)
webknjaz commentedon Jan 25, 2024
@AA-Turner @pradyunsg WDYT about the suggestion above? Could also be an unpacked wheel (
wheel unpack pip-*.whloutput), which feels like a more slim solution, than an sdist.GH-80789: Get rid of the ``ensurepip`` infra for many wheels (#109245)
pythonGH-80789: Get rid of the ``ensurepip`` infra for many wheels (p…