Skip to content

Freeze the _sysconfigdata_* module #131779

Open

Description

@FFY00

Feature or enhancement

Proposal:

After generating _sysconfigdata_*, we could freeze it, like we do to the rest of the standard library.

This should remove the necessity for workarounds such as having to manually set PYTHONPATH to the _sysconfigdata_* directory when running WASM artifacts:

cpython/Tools/wasm/wasi.py

Lines 226 to 227 in 151d1bf

"ENV_VAR_NAME": "PYTHONPATH",
"ENV_VAR_VALUE": f"/{sysconfig_data}",

It should also make it easier to build a fully self-contained statically-linked relocatable interpreter.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Activity

  1. zooba commented on Mar 28, 2025

    @zooba
    Member

    Is it better to freeze it? Or generate native code to extend _sysconfig with the values?

  2. FFY00 commented on Mar 28, 2025

    @FFY00
    MemberAuthor

    I would like to keep things the same with -Xfrozen_modules=off. Unless the freezing implementation gets too complex, I think it would be better to keep it as a module.

  3. zooba commented on Mar 31, 2025

    @zooba
    Member

    I'd expect the build order to get complex - I'm pretty sure _sysconfigdata gets generated after CPython is built, but to freeze it you need to generate it halfway through the build. All the variables must be available at that point, so it's doable, but it might be a pain.

    Then again, having a .py version of the data is no doubt incredibly helpful for some users, as they can get information about an install without having to run it (which may not even be possible). So generating the .py earlier and using the existing freeze infrastructure may be best for everyone.

    Although the frozen modules are there to improve startup time, so they have to balance binary size with likelihood of being imported. I don't know that _sysconfigdata gets loaded every time? If it does, then sure, but if it's only imported on occasion then it might be better to keep it totally separate.

  4. FFY00 commented on Apr 1, 2025

    @FFY00
    MemberAuthor

    I'd expect the build order to get complex - I'm pretty sure _sysconfigdata gets generated after CPython is built, but to freeze it you need to generate it halfway through the build. All the variables must be available at that point, so it's doable, but it might be a pain.

    It's not much of a pain really, it just depends on Makefile and pyconfig.h, so we can use the bootstrap or --with-build-python interpreter to run the script that generates _sysconfigdata_*.py.

    Then again, having a .py version of the data is no doubt incredibly helpful for some users, as they can get information about an install without having to run it (which may not even be possible). So generating the .py earlier and using the existing freeze infrastructure may be best for everyone.

    I am not proposing to remove the .py file. Additionally, we are already shipping a JSON file on POSIX (GH-127302), it shouldn't be difficult to also add this to the Windows build.

    Although the frozen modules are there to improve startup time, so they have to balance binary size with likelihood of being imported. I don't know that _sysconfigdata gets loaded every time? If it does, then sure, but if it's only imported on occasion then it might be better to keep it totally separate.

    Right, that's a good point. It doesn't get loaded every time. Depending on how folks feel about it, we can either use the existing freeze infrastructure, or split it into an extension.

  5. zooba commented on Apr 1, 2025

    @zooba
    Member

    Depending on how folks feel about it, we can either use the existing freeze infrastructure, or split it into an extension.

    We already have a _sysconfig extension, so I'd rather see it "split" into there rather than another extension.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions