Repository navigation
Allow pre-compressed data to be written into zip files via the zipfile module. #113924
Description
Feature or enhancement
Proposal:
Scenario
You have a larger system generating zip files, often they each contain many of the same files, and you've got a big parallel build setup. All of these would benefit from pre-computing, distributing, and caching the compression of individual members going into the zip file.
Proposal
On zipfile.ZipFile write writestr and open(..., mode="w") methods, add arguments allowing the already compressed data to be provided as-is so that the zipfile modules internal compression step is skipped on those.
Has this already been discussed elsewhere?
No response given
Links to previous discussion of this feature:
No response
Activity
serhiy-storchaka commentedon Jan 11, 2024
I thought that it would be more useful to add a method that copies data (one or several entries) from a ZIP file to other ZIP file. It can be used to remove from a ZIP archive (there were several requests for this feature). It can be used to merge several ZIP files in one.
gpshead commentedon Jan 11, 2024
sunrisesarsaparilla commentedon Oct 11, 2024
I've implemented, used, and tested a version of the feature serhiy-storchaka mentioned. My work is built on top of gpshead's PR draft. I'm soliciting feedback for the API design (and any other feedback is also welcome). I've created a discussion here:
https://discuss.python.org/t/add-copy-between-zipfiles-without-recompressing-files/67544
pythongh-113924: add tests for existing exceptions in zipfile stl (py…
pythongh-113924: decouple ZipFile write mode from read mode (python#1…
pythongh-113924: extract method for ZipFile reading logic (python#113924
pythongh-113924: extract method to raise if cannot write to zipfile (p…
pythongh-113924: extract method to setup writing to zipfile (python#1…
pythongh-113924: make ZipFile encrypted files reusable in tests (pyth…
pythongh-113924: add method and tests to copy file from ZipFile (pyth…
pythongh-113924: add method and tests to copy many files from ZipFile (…
pythongh-113924: add method and tests to copy all files from ZipFile (p…
barneygale commentedon Oct 19, 2024
IMO we could start by adding arguments to
ZipFile.open()andZipExtFile.__init__()only, and then keep an eye on what the community builds with them.