setup.py grep_headers_for() is broken by design

@amauryfa

BPO	45598
Nosy	@amauryfa, @abalkin, @tiran, @ned-deily, @merwok, @meadori, @thesamesam
PRs	bpo-45598: Remove ffi header header #29352

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-10-24.12:05:01.584>
labels = ['type-bug', '3.9', '3.10', '3.11', 'ctypes', 'build']
title = 'setup.py grep_headers_for() is broken by design'
updated_at = <Date 2021-11-03.03:37:49.619>
user = 'https://github.com/tiran'

bugs.python.org fields:

activity = <Date 2021-11-03.03:37:49.619>
actor = 'ned.deily'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Build', 'ctypes']
creation = <Date 2021-10-24.12:05:01.584>
creator = 'christian.heimes'
dependencies = []
files = []
hgrepos = []
issue_num = 45598
keywords = ['patch']
message_count = 4.0
messages = ['404926', '404927', '405379', '405580']
nosy_count = 7.0
nosy_names = ['amaury.forgeotdarc', 'belopolsky', 'christian.heimes', 'ned.deily', 'eric.araujo', 'meador.inge', 'thesamesam']
pr_nums = ['29352']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue45598'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

The setup.py helper function grep_headers_for() is used by ctypes build step to search header files for functions. The function kinda works on most platforms, but only by accident. On my system it opens all header files that match /usr/include/*.h and returns true if any (!) header file contains the string (!) "ffi_prep_cif_var". The check would also match "ffi_prep_cif_var" in a comment of an unrelated header file.

We cannot limit the search to "ffi.h" only, because that would break multiarch builds with a header file like this:

    #if defined(__i386__)
    #include "ffi-i386.h"
    #elif defined(__powerpc64__)
    #include "ffi-ppc64.h"
    ...

Solutions:

Use "AC_EGREP_HEADER" autoconf macro. It runs a header file through the C preprocessor
Use AC_LINK_IFELSE autoconf macro. This macro compiles and links code to detect if a library exports a function.
Re-implement any of the macros in pure Python (ugh!)
Just assume that libffi supports ffi_prep_cif_var, ffi_prep_closure_loc, and ffi_closure_alloc on our target platforms. According to [1] the functions have been around for over a decade.

I favor (4).

Ned, what about USING_APPLE_OS_LIBFFI? Is it still relevant?

[1] https://github.com/libffi/libffi/blame/48bdb02867edb7e9f3785ccb4bdff1087fb44246/include/ffi.h.in#L309

If we remove grep_headers_for(), then we can also close bpo-42325

We could look in distro patch trackers to see they patch CPython’s setup.py to evaluate the impact of removing this function.

Ned, what about USING_APPLE_OS_LIBFFI? Is it still relevant?

It was just added a year ago as part of the support for macOS 11 Big Sur and Apple Silicon systems.

Closing as setup.py is now removed, feel free to reopen if required.

Removed 3.9 as it is in security fixes only and this is not a security issue.

Removed 3.9 as it is in security fixes only and this is not a security issue.

Same for 3.10.

setup.py does not exist in main, 3.12, or 3.11; older branches only receive security fixes. Closing as out-of-date.

Navigation Menu

Repository navigation

setup.py grep_headers_for() is broken by design #89761

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

setup.py grep_headers_for() is broken by design #89761

Description

Activity

tiran commented on Oct 24, 2021

tiran commented on Oct 24, 2021

merwok commented on Oct 30, 2021

ned-deily commented on Nov 3, 2021

kumaraditya303 commented on Jul 23, 2022

kumaraditya303 commented on Jul 24, 2022

arhadthedev commented on Apr 29, 2023

erlend-aasland commented on Jun 19, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions