Skip to content

Expand pyatomic coverage #108337

Closed
Closed

Description

@colesbury

Feature or enhancement

Implementing PEP 703 requires use of atomic operations on more data types than provided by pycore_atomic.h. Additionally, pycore_atomic.h is only usable from Py_BUILD_CORE modules; it can't be used in public headers. PEP 703 will require atomic operations in object.h for Py_INCREF/DECREF, for example.

I propose adding a new header to wrap platform/C11 provided atomic operations:

  • The functions will be exposed in the public C-API but prefixed with an underscore to indicate they are private. They need to be exposed because they will be used by other public inline functions.
  • The functions operate on plain data types (i.e. int instead of _Py_atomic_int). This allows use of atomic operations on existing data types where we don't want to change the types of contained fields (i.e, the fields of PyTypeObject.) and avoids some issues regarding mixing C and C++.
  • Use of GCC built-in atomics when available. These are available in GCC 4.8+. They do not require passing -std=gnu11 or -std=c11 to the compiler, which avoids breaking third-party extensions.
  • The memory order is part of the function name instead of an argument. This means there are more function definitions (e.g., both _Py_atomic_load_int and _Py_atomic_load_int_relaxed) but keeps the implementation of each function simpler, particularly for MSVC. This reduces boiler-plate code when stepping through debug builds.

Linked PRs

Activity

  1. added 2 commits that reference this issue on Aug 22, 2023
  2. added 5 commits that reference this issue on Aug 30, 2023
  3. scoder commented on Apr 25, 2024

    @scoder
    Contributor

    The leading underscore marks these as "private". However, with the advent of a GIL-free Python, these will become increasingly useful for third-party code. Why consider them private?

  4. colesbury commented on Apr 25, 2024

    @colesbury
    ContributorAuthor

    There is ongoing discussion about whether they should be made public, but currently no consensus. Some of the reasons to keep them private:

    • When MSVC finally adds support C11 atomics, we may want to use C11 atomics directly or at least simplify the API.
    • Many extensions have better options, such as Rust atomics or C11 or C++11 atomics
    • Some maintainers believe that it is out of scope for CPython: basically that CPython shouldn't be providing cross-platform wrappers for OS or compiler APIs, but limited to focusing on APIs that interact with CPython.

    If we maintained a similar header-only atomics library outside this repo, would that be useful for Cython? Basically, something like https://github.com/python/pythoncapi-compat.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions