Skip to content

ctypes.util incorrectly fails for libraries without DT_SONAME #65821

Closed as not planned
Closed as not planned
ctypes.util incorrectly fails for libraries without DT_SONAME#65821
Labels
OS-unsupportedtopic-ctypestype-bugAn unexpected behavior, bug, or error

Description

@JeremyHuntwork
JeremyHuntwork
mannequin
opened on Jun 1, 2014
BPO 21622
Nosy @vadmium, @ncopa, @thomwiggers, @tianon, @jcastillo2nd
PRs
  • bpo-21622: ctypes.util find_library walk LD_LIBRARY_PATH #10453
  • [2.7] bpo-21622 ctypes.util find_library walk LD_LIBRARY_PATH (GH-10453) #10455
  • bpo-21622: ctypes.util find_library walk LD_LIBRARY_PATH #10460
  • [3.7] bpo-21622 ctypes.util find_library walk LD_LIBRARY_PATH (GH-10460) #10461
  • [2.7] bpo-21622 ctypes.util find_library walk LD_LIBRARY_PATH (GH-10460) #10462
  • bpo-21622: ctypes.util find_library walk LD_LIBRARY_PATH #16940
  • gh-65821: Fix ctypes.util.find_library with musl #18380
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2014-06-01.03:28:22.092>
labels = ['ctypes', 'type-bug']
title = 'ctypes.util incorrectly fails for libraries without DT_SONAME'
updated_at = <Date 2020-04-20.08:40:25.315>
user = 'https://bugs.python.org/JeremyHuntwork'

    bugs.python.org fields:

    activity = <Date 2020-04-20.08:40:25.315>
actor = 'ncopa'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['ctypes']
creation = <Date 2014-06-01.03:28:22.092>
creator = 'Jeremy.Huntwork'
dependencies = []
files = []
hgrepos = []
issue_num = 21622
keywords = ['patch']
message_count = 8.0
messages = ['219481', '264141', '264148', '274361', '274384', '321772', '330668', '366813']
nosy_count = 8.0
nosy_names = ['martin.panter', 'ncopa', 'Jeremy.Huntwork', 'Kylie McClain', 'twiggers', 'Richard Eames', 'tianon', 'jcastillo2nd']
pr_nums = ['10453', '10455', '10460', '10461', '10462', '16940', '18380']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue21622'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6']

    Activity

    1. JeremyHuntwork commented on Jun 1, 2014

      @JeremyHuntwork
      JeremyHuntworkmannequin
      on Jun 1, 2014
      MannequinAuthor

      On my system, the C library (musl) intentionally does not include a SONAME entry.

      This method in particular fails: http://hg.python.org/cpython/file/076705776bbe/Lib/ctypes/util.py#l133

      The function seems to jump through some hoops which may not be necessary. Is there a reason for wanting particularly to use the SONAME entry for the lib?

      In my system the following works as a replacement for _get_soname:

      return os.path.basename(os.path.realpath(f))
    2. JeremyHuntwork
      added
      topic-ctypes
      type-bugAn unexpected behavior, bug, or error
      on Jun 1, 2014

    3. KylieMcClain commented on Apr 25, 2016

      @KylieMcClain
      KylieMcClainmannequin
      on Apr 25, 2016
      Mannequin

      This is still a problem on musl distributions as of 2.7.11. Are there any plans to fix this?

      Patch used by Alpine Linux:
      http://git.alpinelinux.org/cgit/aports/tree/main/python/musl-find_library.patch

    4. vadmium commented on Apr 25, 2016

      @vadmium
      vadmium
      on Apr 25, 2016
      Member

      On Linux, the find_library() function is documented to return “the filename of the library file”, but in reality it seems it return the soname, and therefore breaks if there is no soname. I do not know why we extract the soname, but it has been that way at least since ctypes was added to Python 2.5.

      What do you intend to do with the result of find_library()? See also bpo-9998, especially about searching LD_LIBRARY_PATH. I am struggling to see robust use cases for find_library().

    5. thomwiggers commented on Sep 4, 2016

      @thomwiggers
      thomwiggersmannequin
      on Sep 4, 2016
      Mannequin

      This bug is still present in Python 3.5.

      It breaks, probably among other things, this package: ahupp/python-magic#114.

    6. vadmium commented on Sep 5, 2016

      @vadmium
      vadmium
      on Sep 5, 2016
      Member

      I think it may be reasonable to change the code to return the library file name if no soname can be found.

      In the long term, I think always returning the filename rather than soname might be reasonable. Or even returning the full path, which we tried in bpo-21042 (but rolled back because the solution so far is not consistent across platforms).

    7. tianon commented on Jul 16, 2018

      @tianon
      tianonmannequin
      on Jul 16, 2018
      Mannequin

      This was reported on the Docker image for Python in docker-library/python#111, with the note that it affects the Twisted inotify implementation, so it'd be really neat to see a proper patch in Python (instead of the very musl-/Alpine-assuming patch found in https://github.com/alpinelinux/aports/blob/202f4bea916b0cf974b38ced96ab8fca0b192e3f/main/python2/musl-find_library.patch). <3

    8. jcastillo2nd commented on Nov 29, 2018

      @jcastillo2nd
      jcastillo2ndmannequin
      on Nov 29, 2018
      Mannequin

      The PR 10460 ( for 3.8 ) patches the search attempts to leverage LD_LIBRARY_PATH for the Linux case and catches the case after SONAME, gcc and ldconfig behaviors fail.

      While this may not be set in all environments ( like the musl based Alpine docker image ) setting the environment variable is within the user's control and when properly set does return found libraries.

      PR 10461 and PR 10462 are closed for now, as backports to 2.7 and 3.7 need to have the 3.8 accepted first before being reviewed.

    9. ncopa commented on Apr 20, 2020

      @ncopa
      ncopamannequin
      on Apr 20, 2020
      Mannequin

      I create a PR for this issue.

      It would be nice to have it reviewed.
      #18380

      Thanks!

    10. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    11. serhiy-storchaka
      added this to Ctypes issueson Jan 1, 2024
    12. encukou
      added
      OS-unsupported
      on Mar 8, 2024
    13. github-project-automation
      added this to Unsupported platformson Mar 8, 2024

    14. encukou commented on Feb 23, 2026

      @encukou
      encukou
      on Feb 23, 2026
      Member

      I'm proposing to soft-deprecate ctypes.util.find_library: https://discuss.python.org/t/106232

    15. encukou commented on Mar 20, 2026

      @encukou
      encukou
      on Mar 20, 2026
      Member

      It is now soft-deprecated. It cannot be made to work correctly and consistently on all platforms, and any change runs a chance to break whatever workarounds people already have in place.
      I'm prioritizing stability over new features, and encourage solving this outside ctypes itself.

    16. encukou
      closed this as not plannedon Mar 20, 2026
    17. github-project-automation
      moved this to Done in Ctypes issueson Mar 20, 2026
    18. github-project-automation
      moved this to Done in Unsupported platformson Mar 20, 2026
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      OS-unsupportedtopic-ctypestype-bugAn unexpected behavior, bug, or error

      Projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions