Skip to content

C API: Add guidelines for C APIs with output params #1125

Description

@erlend-aasland

One other issue is whether the output parameter should be touched if there is no meaningful value.

Take PyDict_GetItemSuffixToBeDecided(PyDictObject *op, PyObject *key, PyObject **pvalue).
Should we touch *pvalue if returning -1 or 0.
In other words, in

   PyObject *value;
   int err = PyDict_GetItemSuffixToBeDecided(op, key, &value);

Should value be undefined if err <= 0, or should it be set to NULL?

It is more efficient to not touch it, since the return value specifies that it has no meaning.
But, do we trust all C extension authors not to read the value, which may point to a real object if they are unlucky? Setting *pvalue = NULL is safer.

Originally posted by @markshannon in #1121 (comment)

Activity

  1. erlend-aasland commented on Jun 26, 2023

    @erlend-aasland
    ContributorAuthor

    Quoting Victor's reply, #1121 (comment):

    In my PyDict_GetItemRef() PR, I chose to always initialized *pvalue, especially in the error case (return -1). IMO there is a minor performance overhead, but it closes a whole class of issue: undefined behavior if the function returns -1. Some C compilers try to evaluate all code paths and say that a &value argument "can be undefined" when the code is correct. A common workaround makes me sad: initialize the value in the calling site: PyObject *value = NULL; :-(

    For the "return 0" case (missing key), I also prefer to set *pvalue to NULL for the same reason, but also because people upgrading their code from PyDict_GetItem() and PyDict_GetItemWithError() may want to check if value is NULL or not, rather than checking the return value:

    PyObject *value;  // OMG! undefined value
    if (PyDict_GetItemRef(dict, key, &value) < 0) ... error ...
    else if (value == NULL) ... missing key ...
    else ... key is present ...
    

    Here the function result is not used for the second code path: ... missing key ....

    I'm not comfortable to document that *pvalue is intialized for the error case (return -1). But I wrote an unit test for that.

  2. erlend-aasland commented on Jun 26, 2023

    @erlend-aasland
    ContributorAuthor
  3. erlend-aasland commented on Jun 26, 2023

    @erlend-aasland
    ContributorAuthor

    After thinking about this for some days, I've landed on the conclusion that API exposed in Python.h should explicitly set output params to NULL1; let's try to avoid segfaults bco. API misuse :) I would expect that for most APIs, the minor performance impact would be lost in benchmark noise.

    Footnotes

    1. I agree with Victor that we don't think we need to document that fact in the C API docs. ↩

  4. erlend-aasland commented on Jun 26, 2023

    @erlend-aasland
    ContributorAuthor

    [...] I've landed on the conclusion that API exposed in Python.h should explicitly set output params to NULL1; let's try to avoid segfaults bco. API misuse :) [...]

    IOW, we should ensure that there is no UB when API users check the output parameters.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions