Skip to content

[ctypes][linux] ctypes.util.find_library() crash instead of return None #114257

Closed
Closed
[ctypes][linux] ctypes.util.find_library() crash instead of return None#114257
Labels
3.11only security fixes3.12only security fixes3.13only security fixestopic-ctypestype-bugAn unexpected behavior, bug, or error

Description

@Jacobfaib
Jacobfaib
opened on Jan 18, 2024

Bug report

Bug description:

The problem:

ctypes.util.find_library() raises an exception on certain inputs when it cannot find it.

Expected behavior:

No exception, return None as advertised if it cannot find the library.

Reproducer:

# on python 3.10, but same deal on other versions too
>>> import ctypes.util
>>> ctypes.util.find_library('libgomp')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/path/to/lib/python3.10/ctypes/util.py", line 351, in find_library
    _get_soname(_findLib_gcc(name)) or _get_soname(_findLib_ld(name))
  File "/path/to/lib/python3.10/ctypes/util.py", line 148, in _findLib_gcc
    if not _is_elf(file):
  File "/path/to/lib/python3.10/ctypes/util.py", line 100, in _is_elf
    with open(filename, 'br') as thefile:
FileNotFoundError: [Errno 2] No such file or directory: b'liblibgomp.a'

Note that the crash does not occur if you put nonsense in:

  1. find_library('asdadsas') -> OK
  2. find_library('libasdasdasd') -> OK

The problem appears to be a bit more contrived. Python is attempting to parse the output from gcc, which in this case is:

/usr/lib/gcc/x86_64-linux-gnu/12/../../../x86_64-linux-gnu/Scrt1.o
/usr/lib/gcc/x86_64-linux-gnu/12/../../../x86_64-linux-gnu/crti.o
/usr/lib/gcc/x86_64-linux-gnu/12/crtbeginS.o
/usr/bin/ld: cannot find -llibgomp: No such file or directory
/usr/bin/ld: note to link with /usr/lib/gcc/x86_64-linux-gnu/12/libgomp.a use -l:libgomp.a or rename it to liblibgomp.a
/usr/lib/gcc/x86_64-linux-gnu/12/libgcc.a
/usr/lib/gcc/x86_64-linux-gnu/12/libgcc_s.so
/usr/lib/gcc/x86_64-linux-gnu/12/../../../x86_64-linux-gnu/libgcc_s.so.1
/usr/lib/gcc/x86_64-linux-gnu/12/libgcc.a
collect2: error: ld returned 1 exit status

But then the parser is getting confused trying to parse the useful diagnostic message mentioning libgomp.a.

Machine details (from platform.uname())

uname_result(system='Linux', node='iblis', release='6.5.0-14-generic', version='#14~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Nov 20 18:15:30 UTC 2', machine='x86_64')

CPython versions tested on:

3.10, 3.11, 3.12

Operating systems tested on:

Linux

Linked PRs

Activity

  1. Jacobfaib
    added
    type-bugAn unexpected behavior, bug, or error
    on Jan 18, 2024
  2. Eclips4
    added
    topic-ctypes
    on Jan 18, 2024

  3. Jacobfaib commented on Jan 18, 2024

    @Jacobfaib
    Jacobfaib
    on Jan 18, 2024
    Author

    cc @manopapad

  4. manopapad commented on Jan 18, 2024

    @manopapad
    manopapad
    on Jan 18, 2024

    Full output with some prints (python pulled from conda-forge):

    (test) iblis:~/jacob/legate.core> python --version
Python 3.12.1
(test) iblis:~/jacob/legate.core> cat a.py
import ctypes.util
ctypes.util.find_library('libgomp')

    (test) iblis:~/jacob/legate.core> python a.py
_findLib_gcc(name=libgomp)
expr=b'[^\\(\\)\\s]*liblibgomp\\.[^\\(\\)\\s]*'
trace=b'/usr/lib/gcc/x86_64-linux-gnu/12/../../../x86_64-linux-gnu/Scrt1.o\n/usr/lib/gcc/x86_64-linux-gnu/12/../../../x86_64-linux-gnu/crti.o\n/usr/lib/gcc/x86_64-linux-gnu/12/crtbeginS.o\n/usr/bin/ld: cannot find -llibgomp: No such file or directory\n/usr/bin/ld: note to link with /usr/lib/gcc/x86_64-linux-gnu/12/libgomp.a use -l:libgomp.a or rename it to liblibgomp.a\n/usr/lib/gcc/x86_64-linux-gnu/12/libgcc.a\n/usr/lib/gcc/x86_64-linux-gnu/12/libgcc_s.so\n/usr/lib/gcc/x86_64-linux-gnu/12/../../../x86_64-linux-gnu/libgcc_s.so.1\n/usr/lib/gcc/x86_64-linux-gnu/12/libgcc.a\ncollect2: error: ld returned 1 exit status\n'
res=[b'liblibgomp.a']
Traceback (most recent call last):
  File "/home/mpapadakis/jacob/legate.core/a.py", line 2, in <module>
    ctypes.util.find_library('libgomp')
  File "/home/mpapadakis/mambaforge/envs/test/lib/python3.12/ctypes/util.py", line 355, in find_library
    _get_soname(_findLib_gcc(name)) or _get_soname(_findLib_ld(name))
                ^^^^^^^^^^^^^^^^^^
  File "/home/mpapadakis/mambaforge/envs/test/lib/python3.12/ctypes/util.py", line 152, in _findLib_gcc
    if not _is_elf(file):
           ^^^^^^^^^^^^^
  File "/home/mpapadakis/mambaforge/envs/test/lib/python3.12/ctypes/util.py", line 100, in _is_elf
    with open(filename, 'br') as thefile:
         ^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: b'liblibgomp.a'

    I assume the double "lib" is intentional, and this call was meant to fail, and just ended up mis-parsing an unrelated part of the output diagnostic message.

  5. aisk commented on Jan 21, 2024

    @aisk
    aisk
    on Jan 21, 2024
    Member

    The find_library function will invoke gcc and search for any matches to the pattern lib{name} (in this case, liblibgomp) in its output.

    In the new version of gcc, it will display a recommendation tip in case you add the unnecessary lib prefix, like:

    /usr/bin/ld: note to link with /usr/lib/gcc/x86_64-linux-gnu/12/libgomp.a use -l:libgomp.a or rename it to liblibgomp.a

    If a matching result, such as liblibgomp.a, is found, it will be used as an argument to _is_elf. Unfortunately, if the file does not exist or for any other reason when it's opened, it will raise an error.

    I'm not sure if any other OSError should be ignored, but at least FileNotFoundError should be ignored in this case.

  6. aisk
    added a commit that references this issue on Jan 21, 2024

    pythongh-114257: Ignore the `FileNotFound` error in `ctypes.util._is_…

    31701e1
  7. serhiy-storchaka
    added
    3.11only security fixes
    3.12only security fixes
    3.13only security fixes
    on Jan 22, 2024
  8. serhiy-storchaka
    added a commit that references this issue on Jan 22, 2024

    gh-114257: Ignore the FileNotFound error in ctypes.util._is_elf() (GH…

    7fc51c3
  9. miss-islington
    added 2 commits that reference this issue on Jan 22, 2024

    pythongh-114257: Ignore the FileNotFound error in ctypes.util._is_elf…

    d256c8c

    pythongh-114257: Ignore the FileNotFound error in ctypes.util._is_elf…

    736c0cb
  10. serhiy-storchaka
    added 2 commits that reference this issue on Jan 22, 2024

    [3.12] gh-114257: Ignore the FileNotFound error in ctypes.util._is_el…

    ed567c1

    [3.11] gh-114257: Ignore the FileNotFound error in ctypes.util._is_el…

    f5d9980
  11. serhiy-storchaka
    closed this as completedon Jan 22, 2024
  12. aisk
    added a commit that references this issue on Feb 11, 2024

    pythongh-114257: Ignore the FileNotFound error in ctypes.util._is_elf…

    79c7dec
  13. Glyphack
    added a commit that references this issue on Sep 2, 2024

    pythongh-114257: Ignore the FileNotFound error in ctypes.util._is_elf…

    0860b6b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.11only security fixes3.12only security fixes3.13only security fixestopic-ctypestype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions