Skip to content

ZipFile.writestr should respect SOURCE_DATE_EPOCH #91279

Closed
Closed
ZipFile.writestr should respect SOURCE_DATE_EPOCH#91279
Labels
3.10 (EOL)end of life3.11only security fixes3.9 (EOL)end of lifestdlibStandard Library Python modules in the Lib/ directorytype-featureA feature request or enhancement

Description

@SomberNight
SomberNight
mannequin
opened on Mar 25, 2022
BPO 47123
Nosy @SomberNight
Files
  • zipfile_respect_sourcedate.diff: change the ZipFile.writestr to respect SOURCE_DATE_EPOCH
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2022-03-25.19:47:10.113>
labels = ['type-feature', 'library', '3.9', '3.10', '3.11']
title = 'ZipFile.writestr should respect SOURCE_DATE_EPOCH'
updated_at = <Date 2022-03-25.19:47:10.113>
user = 'https://github.com/SomberNight'

    bugs.python.org fields:

    activity = <Date 2022-03-25.19:47:10.113>
actor = 'ghost43'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2022-03-25.19:47:10.113>
creator = 'ghost43'
dependencies = []
files = ['50702']
hgrepos = []
issue_num = 47123
keywords = ['patch']
message_count = 1.0
messages = ['416015']
nosy_count = 1.0
nosy_names = ['ghost43']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue47123'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

    Linked PRs

    Activity

    1. SomberNight commented on Mar 25, 2022

      @SomberNight
      SomberNightmannequin
      on Mar 25, 2022
      MannequinAuthor

      Currently ZipFile.writestr writes the local time into the ZipFile.
      (depends on both current time and local timezone)
      See

      cpython/Lib/zipfile.py

      Lines 1816 to 1817 in 20e6e56

      zinfo = ZipInfo(filename=zinfo_or_arcname,
      date_time=time.localtime(time.time())[:6])

      This makes pip installing a package generate non-reproducible build artifacts.

      Specifically, Scripts/*.exe files (created for packages that define entry_points/console_scripts) are not reproducible on Windows when installed by pip. This also leaks into the *.dist-info/RECORD files.

      For example, after running pip install wheel or pip install pyinstaller,
      in wheel-0.37.1.dist-info/RECORD, I have this line:

      ../../Scripts/wheel.exe,sha256=u9TbPw2XNs_F9uy7y2zwumuzAZDbOSB7BXjLHZ0tTHg,97103

      in pyinstaller-4.10.dist-info/RECORD, I have these lines:

      ../../Scripts/pyi-archive_viewer.exe,sha256=nC-9muPlIhUC1qvFkXHpyKJyRQqXISXxbUPXQ1XVOiM,97133
../../Scripts/pyi-bindepend.exe,sha256=udFHiAdndPpSwaIqmhmLEy36IUs1cNNoNQznSEnLJQQ,97128
../../Scripts/pyi-grab_version.exe,sha256=3ET9E841tFWujFL99aG4frzgwlP9f9pAkMgE0k2UGK0,97131
../../Scripts/pyi-makespec.exe,sha256=dJkfmITdLJhyPngmqziqqj5tH9qqfeQc5BTubeoXWUs,97127
../../Scripts/pyi-set_version.exe,sha256=sWmcOVS93fUY-wbdoz6ixBCvjy1tC4Aaw30DMmrmo-0,97130
../../Scripts/pyinstaller.exe,sha256=haInbhH0pImJn24cW4v917oUZmzXZj8OE89KFh4MO2Y,97112

      Upon comparing multiple Scripts/wheel.exe files, I've found that the only difference is due to the above-mentioned timestamp embedded inside the exe (or rather, same timestamp embedded twice).

      The exe files get created by distlib (vendored by pip).
      Here is a traceback with an artificial exception to illustrate the codepath:

      (env) PS C:\tmp> pip install --no-build-isolation pyinstaller
Collecting pyinstaller
  Using cached pyinstaller-4.10-py3-none-win_amd64.whl (2.0 MB)
Requirement already satisfied: setuptools in c:\tmp\env\lib\site-packages (from pyinstaller) (61.0.0)
Requirement already satisfied: pyinstaller-hooks-contrib>=2020.6 in c:\tmp\env\lib\site-packages (from pyinstaller) (2022.3)
Requirement already satisfied: altgraph in c:\tmp\env\lib\site-packages (from pyinstaller) (0.17.2)
Requirement already satisfied: pefile>=2017.8.1 in c:\tmp\env\lib\site-packages (from pyinstaller) (2021.9.3)
Requirement already satisfied: pywin32-ctypes>=0.2.0 in c:\tmp\env\lib\site-packages (from pyinstaller) (0.2.0)
Requirement already satisfied: future in c:\tmp\env\lib\site-packages (from pefile>=2017.8.1->pyinstaller) (0.18.2)
Installing collected packages: pyinstaller
ERROR: Exception:
Traceback (most recent call last):
  File "C:\tmp\env\lib\site-packages\pip\_internal\cli\base_command.py", line 167, in exc_logging_wrapper
    status = run_func(*args)
  File "C:\tmp\env\lib\site-packages\pip\_internal\cli\req_command.py", line 205, in wrapper
    return func(self, options, args)
  File "C:\tmp\env\lib\site-packages\pip\_internal\commands\install.py", line 405, in run
    installed = install_given_reqs(
  File "C:\tmp\env\lib\site-packages\pip\_internal\req\__init__.py", line 73, in install_given_reqs
    requirement.install(
  File "C:\tmp\env\lib\site-packages\pip\_internal\req\req_install.py", line 769, in install
    install_wheel(
  File "C:\tmp\env\lib\site-packages\pip\_internal\operations\install\wheel.py", line 729, in install_wheel
    _install_wheel(
  File "C:\tmp\env\lib\site-packages\pip\_internal\operations\install\wheel.py", line 646, in _install_wheel
    generated_console_scripts = maker.make_multiple(scripts_to_generate)
  File "C:\tmp\env\lib\site-packages\pip\_vendor\distlib\scripts.py", line 440, in make_multiple
    filenames.extend(self.make(specification, options))
  File "C:\tmp\env\lib\site-packages\pip\_internal\operations\install\wheel.py", line 427, in make
    return super().make(specification, options)
  File "C:\tmp\env\lib\site-packages\pip\_vendor\distlib\scripts.py", line 429, in make
    self._make_script(entry, filenames, options=options)
  File "C:\tmp\env\lib\site-packages\pip\_vendor\distlib\scripts.py", line 329, in _make_script
    self._write_script(scriptnames, shebang, script, filenames, ext)
  File "C:\tmp\env\lib\site-packages\pip\_vendor\distlib\scripts.py", line 263, in _write_script
    raise Exception(f"heyheyhey2. {sha256(launcher)=}. {sha256(shebang)=}. {sha256(zip_data)=}. " +
Exception: heyheyhey2. sha256(launcher)='a00a877acefc'. sha256(shebang)='58628e924f22'. sha256(zip_data)='a423496a0482'. ('SOURCE_DATE_EPOCH' in os.environ)=True

      The interesting code is here:
      https://github.com/pypa/distlib/blob/d0e3f49df5d1aeb9daeaaabf0391c9e13e4a6562/distlib/scripts.py#L251-L252
      This calls into the cpython standard library, where time.time() gets written into the file:

      cpython/Lib/zipfile.py

      Lines 1816 to 1817 in 20e6e56

      zinfo = ZipInfo(filename=zinfo_or_arcname,
      date_time=time.localtime(time.time())[:6])

      Ideally, either distlib or the stdlib zipfile module should be changed to respect SOURCE_DATE_EPOCH, but it's not entirely clear to me which...

      The attached patch changes ZipFile.writestr to respect SOURCE_DATE_EPOCH if set.

      related pypa/distlib#164
      related spesmilo/electrum#7739

    2. SomberNight
      added
      3.9 (EOL)end of life
      3.10 (EOL)end of life
      3.11only security fixes
      stdlibStandard Library Python modules in the Lib/ directory
      type-featureA feature request or enhancement
      on Mar 25, 2022
    3. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    4. serhiy-storchaka
      added this to @serhiy-storchaka's project python-zipfileon May 8, 2022
    5. serhiy-storchaka
      added this to Zipfile issues 📦on May 19, 2022
    6. jaraco
      added a commit that references this issue on Jan 20, 2025

      gh-91279: ZipFile.writestr now respect SOURCE_DATE_EPOCH (#124435)

      5d57959
    7. srinivasreddy
      added a commit that references this issue on Jan 21, 2025

      pythongh-91279: ZipFile.writestr now respect SOURCE_DATE_EPOCH (pytho…

      a3b40da
    8. Eclips4
      closed this as completedon Jan 22, 2025
    9. github-project-automation
      moved this to Done in @serhiy-storchaka's project python-zipfileon Jan 22, 2025
    10. github-project-automation
      moved this to Done in Zipfile issues 📦on Jan 22, 2025
    11. vstinner
      added a commit that references this issue on Mar 20, 2026

      gh-91279: Note `SOURCE_DATE_EPOCH` support in `ZipFile.writestr()` doc (

      5ad738f
    12. miss-islington
      added a commit that references this issue on Mar 20, 2026

      pythongh-91279: Note `SOURCE_DATE_EPOCH` support in `ZipFile.writestr…

      9cec510
    13. vstinner
      added a commit that references this issue on Mar 20, 2026

      [3.14] gh-91279: Note `SOURCE_DATE_EPOCH` support in `ZipFile.writest…

      8b447fb
    14. CuriousLearner
      added a commit that references this issue on Mar 22, 2026

      Merge remote-tracking branch 'upstream/main' into fix-pythongh-138577

      2691ad9
    15. ljfp
      added a commit that references this issue on Apr 25, 2026

      pythongh-91279: Note `SOURCE_DATE_EPOCH` support in `ZipFile.writestr…

      7cb649a
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.10 (EOL)end of life3.11only security fixes3.9 (EOL)end of lifestdlibStandard Library Python modules in the Lib/ directorytype-featureA feature request or enhancement

      Projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions