Skip to content

Implement PEP 784 - Adding Zstandard to the Python standard library #132983

Description

@emmatyping

Feature or enhancement

This is a tracking issue for implementing PEP 784. See the PEP text for more details.

Since the diff is significant (~10k lines) I wanted to split up the PRs a bit.

Implementation Plan:

  • Add compression module just re-exporting existing compression modules. Move the _compression module.
  • Add _zstd native module with Unix build config
  • Add Windows build config for _zstd (blocked on adding libzstd to cpython-source-deps) and SBOM config.
  • Add zstd Python module with tests
  • add NEWS/What's New section
  • Add documentation for zstd
  • Decide on whether or not to check parameter types for options
  • Verify style conformance of C and Python code
    • Python
    • C
  • Refactor _train_dict and _finalize_dict to share common code
  • Fix test_zstd failed on ubuntu with free-threading #133885
  • Improve error message when options value can't be converted to int

Linked PRs

Activity

  1. added
    stdlibStandard Library Python modules in the Lib/ directory
    type-featureA feature request or enhancement
    on Apr 26, 2025
  2. self-assigned this
    on Apr 26, 2025
  3. picnixz commented on Apr 26, 2025

    @picnixz
    Member

    It's something I should have asked on discourse, but is compression the final name of the public module? I see in the PEP that we want to mimic hashlib.algorithms_guaranteed as well and one way to mitigate the name shadowing is to actually use compressionlib as the name (we have hashlib, annotationlib, and we could have compressionlib as well; I also think compressionlib is a less likely name to be chosen but it's not the best one as well). Feel free to ignore this question as the PEP has been accepted.

  4. emmatyping commented on Apr 26, 2025

    @emmatyping
    MemberAuthor

    It's something I should have asked on discourse, but is compression the final name of the public module?

    Yes, the decision was to use compression. A search of Github code showed that it was used very infrequently. Literally one in a million files https://discuss.python.org/t/pep-784-adding-zstandard-to-the-standard-library/87377/66

  5. added a commit that references this issue on Apr 27, 2025
  6. added a commit that references this issue on Apr 28, 2025
  7. added 2 commits that reference this issue on Apr 28, 2025
  8. added a commit that references this issue on May 4, 2025
  9. AA-Turner commented on May 4, 2025

    @AA-Turner
    Member

    I've had a go at adding support for building on Windows in #133366, let me know thoughts.

    A

  10. ned-deily commented on May 5, 2025

    @ned-deily
    Member
  11. 172 remaining items

  12. added a commit that references this issue on Aug 7, 2025
  13. added 4 commits that reference this issue on Aug 19, 2025
  14. added 3 commits that reference this issue on Sep 9, 2025
  15. added 7 commits that reference this issue on Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

3.14bugs and security fixesextension-modulesC modules in the Modules dirstdlibStandard Library Python modules in the Lib/ directorytype-featureA feature request or enhancement

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions