Skip to content

Provide a variant of PyDict_SetDefault that returns a new reference (instead of a borrowed reference) #112066

Description

@colesbury

The PyDict_SetDefault(mp, key, defaultobj) function returns a borrowed reference to the value corresponding to key. This poses a thread-safety issue particularly for the case where key is already in the dict. In the --disable-gil builds, the returned value may no longer be valid if another thread concurrently modifies the dict.

Proposal (from Victor)

int PyDict_SetDefaultRef(PyObject *dict, PyObject *key, PyObject *default_value, PyObject **value);

The **value pointer is optional. If it is NULL, it is not used.

  • If the key is present in the dict, set *value to a new reference to the current value (if value is not NULL), and return 1.
  • If the key is missing from the dict, insert the key and default_value, and set *value to a new reference to default_value (if value is not NULL), and return 0.
  • On error, set *value to NULL if value is not NULL, and return -1.

Ideally, this new function would be public and part of the stable ABI so that it could be used by all extensions, but even an internal-only function would unblock some of the nogil changes.

EDIT: Updated with @vstinner's proposal

Linked PRs

Activity

  1. colesbury commented on Nov 14, 2023

    @colesbury
    ContributorAuthor
  2. vstinner commented on Nov 14, 2023

    @vstinner
    Member

    Following previous API such as PyDict_GetItemRef() and PyDict_Pop(), I propose the following API:

    int PyDict_SetDefaultRef(PyObject *dict, PyObject *key, PyObject *default_value, **value)
    
    • If key is present in dict, set *value to a new reference to the current value if value is not NULL, and return 1.
    • If key is missing in dict, set key to default_value in dict, set *value to a new reference to the default_value if value is not NULL, and return 0.
    • On error, set *value to NULL if value is not NULL, and return -1.

    So value can be NULL if the value is not used.

    See also API evolution: Return value conventions.

    cc @zooba @serhiy-storchaka @erlend-aasland

  3. colesbury commented on Nov 14, 2023

    @colesbury
    ContributorAuthor

    @vstinner - sounds good. I updated the issue description with your proposal.

  4. vstinner commented on Nov 14, 2023

    @vstinner
    Member

    Do you want to propose a PR?

  5. added a commit that references this issue on Nov 15, 2023
  6. self-assigned this
    on Nov 15, 2023
  7. added a commit that references this issue on Nov 17, 2023
  8. added 3 commits that reference this issue on Feb 6, 2024
  9. added 2 commits that reference this issue on Feb 14, 2024
  10. added a commit that references this issue on May 7, 2024
  11. added a commit that references this issue on May 17, 2024
  12. added a commit that references this issue on May 22, 2024
  13. added a commit that references this issue on May 22, 2024
  14. added a commit that references this issue on Jul 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions