Skip to content

uuid module build fix on FreeBSD proposal #85077

Open
Open
uuid module build fix on FreeBSD proposal#85077
Labels
3.11only security fixes3.12only security fixes3.13only security fixesOS-freebsdbuildThe build process and cross-build

Description

@devnexen
devnexen
mannequin
opened on Jun 7, 2020
BPO 40900
Nosy @tiran, @koobs, @devnexen, @TIGirardi, @dbaio
PRs
  • gh-85077: uuid module build fix on FreeBSD proposal. #20693
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2020-06-07.12:55:47.007>
labels = ['build', '3.10']
title = 'uuid module build fix on FreeBSD proposal'
updated_at = <Date 2020-09-16.23:51:22.988>
user = 'https://github.com/devnexen'

    bugs.python.org fields:

    activity = <Date 2020-09-16.23:51:22.988>
actor = 'dbaio'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Build', 'FreeBSD']
creation = <Date 2020-06-07.12:55:47.007>
creator = 'devnexen'
dependencies = []
files = []
hgrepos = []
issue_num = 40900
keywords = []
message_count = 5.0
messages = ['370906', '370907', '370953', '371158', '371472']
nosy_count = 5.0
nosy_names = ['christian.heimes', 'koobs', 'devnexen', 'TIGirardi', 'dbaio']
pr_nums = ['20693']
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue40900'
versions = ['Python 3.10']

    Activity

    1. devnexen
      added
      3.10 (EOL)end of life
      buildThe build process and cross-build
      on Jun 7, 2020

    2. tiran commented on Jun 7, 2020

      @tiran
      tiran
      on Jun 7, 2020
      Member

      Please explain which problem you are facing and how your proposal is going to fix the problem.

    3. devnexen commented on Jun 7, 2020

      @devnexen
      devnexenmannequin
      on Jun 7, 2020
      MannequinAuthor

      This s about header picked up in a certain order. In case of FreeBSD, the uui_create case is taken which comes from the <uuid.h> but ... <uuid/uuid.h> is detected too.

    4. koobs commented on Jun 8, 2020

      @koobs
      koobs
      on Jun 8, 2020

      FreeBSD base provides uuid.h (uuid(3)) but uuid libraries/headers can be provided by e2fsprogs-libuuid (for example) in another location, for example /usr/local/

      Pythons build system doesn't provide sufficient granularity to pass include/library locations for *specific* components of the build in every circumstance.

      The following is a bug related to Python 3.7+ and the uuid.h / linking problem:

      https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229562

      The current patch just ignores the third-party (/usr/local) version of uuid.h, allowing the build to proceed with the headers provided by base:

      https://bz-attachments.freebsd.org/attachment.cgi?id=200603

      The proper/permanent solution might be to allow a --with-uuid=<location> style build argument

      This issue of conflicting / multiple library versions and include order has also come up in various forms for other third party library support in Python, such as gettext (libintl), expat, ssl, libffi, curses)

      See the following FreeBSD Python port block for the libintl example:

      https://svnweb.freebsd.org/ports/head/lang/python37/Makefile?revision=536776&view=markup#l71

    5. koobs commented on Jun 10, 2020

      @koobs
      koobs
      on Jun 10, 2020

      Another example, this time for lzma:

      https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209355

    6. TIGirardi commented on Jun 13, 2020

      @TIGirardi
      TIGirardimannequin
      on Jun 13, 2020
      Mannequin

      The problem isn't exclusive to FreeBSD. There are at least 3 problems here (see also: bpo-32627).

      -First is that there are 2 low-level uuid interfaces (3 if you count Windows but that is isolated and not a problem here) supposed to be declared on the headers below:

      1. uuid/uuid.h which is supposed to maybe declare
        uuid_generate_time_safe and declare uuid_generate_time, and probably must be linked with '-luuid'.

      2. uuid.h: which is supposed to declare uuid_create and maybe uuid_enc_be (which as I understand is libc on FreeBSD an some others).

      The way it currently works (I guess, 'configure' conspicuously doesn't end in '.py'), 'configure' tries to find uuid/uuid.h, if it succeeds, it #define HAVE_UUID_UUID_H 1, then it tries to find 'uuid.h', which on success #define HAVE_UUID_H 1.

      Then it tries to compile with #include <uuid/uuid.h> and getting the address of uuid_generate_time_safe. On success, #define HAVE_UUID_GENERATE_TIME_SAFE 1.

      Next, #include <uuid.h> and first tries to compile a code that gets uuid_create's adress (on success #define HAVE_UUID_CREATE 1), then compile to get uuid_enc_be adress (on success #define HAVE_UUID_ENC_BE 1).

      The last two ignores header detection information.

      -The second problem is on make build and 'setup.py':

      Then 'setup.py' proceeds to ignore all that and tries to find 'uuid.h' on Py_build_ext(dist).inc_dirs or in '/usr/include/uuid', which doesn't work if the user has 'uuid/uuid.h' in inc_dirs but not in '/usr/include/uuid'. (It should get the macros from sysconfig and see which header it is looking for and not put '/usr/include/uuid' on the search path, 'configure' already found or not and already disabled the include if not.)

      If it finds it, it tries to find the uuid library, and adds it to linking on success, else assumes it doesn't need to do anything else and succeeds.

      If it doesn't find the header (which it may not, even if 'configure' did it), it gives up on the module and warns the user in the end of make build.

      -Third, Modules/_uuidmodule.c (see bpo-32627):

      Inspect the #includes and py_uuid_generate_time_safe() and you can probably guess how this happens, and you can probably see another bug waiting to to happen: neither 'configure', nor 'setup.py' actually do what '_uuidmodule.c' is expecting, and py_uuid_generate_time_safe() will fallback to uuid_generate_time() which may not be declared (which gives you another bug, but on linking time, because you may not have passed '-luuid').

      At least on Linux (CentOS) on a custom --prefix, with custom libuuid.so, this proccess fails on Python 3.8.3. I edited 'setup.py' to correctly find the header and the library (for my case, anyway, I have no idea on FreeBSD) and it worked. (You don't support this use case, but that's another issue, just adding more info.)

      OP's patch (which appears to be to check the functions only on detection of the respective modules) may work for his specific case, but I suspect that for a more stable solution we need to change _uuidmodule.c and at least one of 'configure(.ac)' and 'setup.py':

      • the module directives should behave as 'configure' tests;
      • which should ideally test for uuid_generate_time();
        -'setup.py' should listen to 'configure' the same way the module directives do.

      (and by 'we' I mean I'm volunteering to help, but I will need help on that).

      Maybe what I'm saying should be another issue entirely, though I'm adding build to components (and I think we should include 3.8 and 3.9 too). Please advise.

    7. TIGirardi
      added
      buildThe build process and cross-build
      on Jun 13, 2020
    8. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    9. vstinner
      added
      OS-freebsd
      on Apr 21, 2022

    10. serhiy-storchaka commented on Dec 26, 2023

      @serhiy-storchaka
      serhiy-storchaka
      on Dec 26, 2023
      Member

      Please check that this issue is still actual. There were many changes in the configure script, in particularly #29741 changed the same uuid detection code.

    11. serhiy-storchaka
      added
      3.11only security fixes
      3.12only security fixes
      3.13only security fixes
      and removed
      3.10 (EOL)end of life
      on Dec 26, 2023
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.11only security fixes3.12only security fixes3.13only security fixesOS-freebsdbuildThe build process and cross-build

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions