Skip to content

Keep ob_tid, ob_ref_local, and ob_ref_shared fields valid across allocations in free-threaded build #115491

Closed
Closed
Keep ob_tid, ob_ref_local, and ob_ref_shared fields valid across allocations in free-threaded build#115491
Assignees
colesbury
Labels
topic-free-threadingtype-featureA feature request or enhancement

Description

@colesbury
colesbury
opened on Feb 14, 2024

Feature or enhancement

The free-threaded implementation of dict and list try avoid acquiring locks during read operations. To support this, we need to be able to access the reference count fields of Python objects after they are deallocated (and possibly reallocated). Some of this support is provided by #115103. Additionally, we need to ensure that the debug allocators do not overwrite these fields with "dead" bytes 0xDD, which might make the object look "alive" by having a non-zero reference count.

We still would like to overwrite the rest of the allocation (i.e., from ob_type onwards) to detect use-after-frees in debug builds.

See also: https://peps.python.org/pep-0703/#optimistically-avoiding-locking

Linked PRs

Activity

  1. colesbury
    added
    type-featureA feature request or enhancement
    topic-free-threading
    on Feb 14, 2024
  2. colesbury
    self-assigned this
    on Feb 15, 2024
  3. colesbury
    added 4 commits that reference this issue on Feb 16, 2024

    pythongh-115491: Keep some fields valid across allocations (free-thre…

    58a2d1f

    pythongh-115491: Keep some fields valid across allocations in obmallo…

    20c0c81

    gh-115491: Keep some fields valid across allocations (free-threading) (…

    cc82e33

    pythongh-115491: Keep some fields valid across allocations in obmallo…

    85b3647
  4. DinoV
    added a commit that references this issue on Feb 21, 2024

    gh-115491: Keep some fields valid across allocations in obmalloc (fre…

    347acde

  5. colesbury commented on Feb 21, 2024

    @colesbury
    colesbury
    on Feb 21, 2024
    ContributorAuthor

    This still needs the mimalloc support in #115103 to fully preserve the fields, but the changes to filling debug bytes is done now.

  6. colesbury
    closed this as completedon Feb 21, 2024
  7. colesbury
    added a commit that references this issue on Feb 29, 2024

    pythongh-115491: Fix compiler warning on macOS

    36a959e
  8. DinoV
    added a commit that references this issue on Mar 1, 2024

    gh-115491: Fix Clang compiler warning (#116153)

    d7ddd90
  9. woodruffw
    added 3 commits that reference this issue on Mar 4, 2024

    pythongh-115491: Keep some fields valid across allocations (free-thre…

    be2dcd0

    pythongh-115491: Keep some fields valid across allocations in obmallo…

    041aa45

    pythongh-115491: Fix Clang compiler warning (python#116153)

    625d530
  10. adorilson
    added a commit that references this issue on Mar 25, 2024

    pythongh-115491: Fix Clang compiler warning (python#116153)

    18be625
  11. diegorusso
    added 3 commits that reference this issue on Apr 17, 2024

    pythongh-115491: Keep some fields valid across allocations (free-thre…

    de74b97

    pythongh-115491: Keep some fields valid across allocations in obmallo…

    6e942bd

    pythongh-115491: Fix Clang compiler warning (python#116153)

    b1dd0bb
  12. LukasWoodtli
    added 2 commits that reference this issue on Jan 22, 2025

    pythongh-115491: Keep some fields valid across allocations (free-thre…

    0bb51fd

    pythongh-115491: Keep some fields valid across allocations in obmallo…

    9720a85
  13. corona10
    added 2 commits that reference this issue on Oct 10, 2026

    Re-apply CPython-specific changes to mimalloc v2.5.2

    fa3a143

    Re-apply CPython-specific changes to mimalloc v2.5.2

    836dced
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

topic-free-threadingtype-featureA feature request or enhancement

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions