Skip to content

[C API] Enhance PyErr_WriteUnraisable() API to pass an error message #108082

Closed
Closed
[C API] Enhance PyErr_WriteUnraisable() API to pass an error message#108082
Labels
topic-C-APItype-featureA feature request or enhancement

Description

@vstinner
vstinner
opened on Aug 17, 2023

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Proposal:

I added a private _PyErr_WriteUnraisableMsg() API but it cannot be used in third party code, nor in stdlib extensions which try to avoid private/internal functions (like sqlite3). Moreover, _PyErr_WriteUnraisableMsg() API doesn't allow to fully customize the error message.

The limitation of _PyErr_WriteUnraisableMsg() affected PR #106674 which has to add ; consider using ... in the error message which is not great.

        _PyErr_WriteUnraisableMsg(
            "in PyMapping_HasKeyString(); consider using "
            "PyMapping_GetOptionalItemString() or PyMapping_GetItemString()",
            NULL);

@serhiy-storchaka suggested to add an API which allows string formatting, similar to PyErr_Format().

_PyErr_WriteUnraisableMsg() was added in issue #81010.

By the way, we should go through all calls to PyErr_WriteUnraisable() and add more context to more these logs easier to get for developers: give more context about what the issue is, how to fix it, where it occurred, etc.

Linked PRs

Activity

  1. vstinner
    added
    type-featureA feature request or enhancement
    topic-C-API
    on Aug 17, 2023

  2. serhiy-storchaka commented on Aug 17, 2023

    @serhiy-storchaka
    serhiy-storchaka
    on Aug 17, 2023
    Member

    I already started to work on this, but was not sure about API, so left it to your return.

    I initially was going to add simple function:

    void PyErr_FormatUnraisable(const char *format, ...);

    With special case PyErr_FormatUnraisable(NULL, obj) equal to PyErr_WriteUnraisable(obj).

    But it now creates the UnraisableHookArgs object which has field object. So we need more complex API:

    void PyErr_FormatUnraisable(PyObject *obj, const char *format, ...);

    With PyErr_WriteUnraisable(obj) equal to PyErr_FormatUnraisable(obj, "Exception ignored in: %R", obj) or special case PyErr_FormatUnraisable(obj, NULL).

    Or we always set UnraisableHookArgs.object to None if use PyErr_FormatUnraisable(), then we can make the use simpler. Then custom message and object will be mutually exclusive, but seems that currently most of cases with custom message pass NULL as object.

    Or we can keep PyErr_FormatUnraisable() simpler and add a separate function PyErr_FormatUnraisableObject() which allows to specify both custom message and object.

    void PyErr_FormatUnraisable(const char *format, ...);
void PyErr_FormatUnraisableObject(PyObject *obj, const char *format, ...);

  3. vstinner commented on Sep 4, 2023

    @vstinner
    vstinner
    on Sep 4, 2023
    MemberAuthor

    I wrote PR #108863 to remove the private _PyErr_WriteUnraisableMsg() function from the public C API.

  4. serhiy-storchaka
    added a commit that references this issue on Oct 19, 2023

    pythongh-108082: Add PyErr_FormatUnraisable() function

    de02658
  5. serhiy-storchaka
    added 2 commits that reference this issue on Oct 29, 2023

    pythongh-108082: C API: Add tests for PyErr_WriteUnraisable()

    93da12c

    gh-108082: C API: Add tests for PyErr_WriteUnraisable() (GH-111455)

    bca3305
  6. miss-islington
    added a commit that references this issue on Oct 30, 2023

    pythongh-108082: C API: Add tests for PyErr_WriteUnraisable() (python…

    88d383b
  7. serhiy-storchaka
    added 4 commits that reference this issue on Oct 30, 2023

    [3.12] gh-108082: C API: Add tests for PyErr_WriteUnraisable() (GH-11…

    748bc48

    gh-108082: Add PyErr_FormatUnraisable() function (GH-111086)

    f6a0232

    pythongh-108082: Use PyErr_FormatUnraisable()

    e126cd5

    gh-108082: Use PyErr_FormatUnraisable() (GH-111580)

    970e719

  8. serhiy-storchaka commented on Nov 2, 2023

    @serhiy-storchaka
    serhiy-storchaka
    on Nov 2, 2023
    Member

    Few calls of _PyErr_WriteUnraisableMsg are left which pass both message and non-NULL object.

    Modules/atexitmodule.c:140:            _PyErr_WriteUnraisableMsg("in atexit callback", the_func);
--
Modules/_threadmodule.c:1074:            _PyErr_WriteUnraisableMsg("in thread started by", boot->func);
--
Modules/_ctypes/callbacks.c:219:        _PyErr_WriteUnraisableMsg("on calling ctypes callback function",
Modules/_ctypes/callbacks.c-220-                                  callable);
--
Modules/_ctypes/callbacks.c:261:            _PyErr_WriteUnraisableMsg("on converting result "
Modules/_ctypes/callbacks.c-262-                                      "of ctypes callback function",
Modules/_ctypes/callbacks.c-263-                                      callable);
--
Modules/_ctypes/callbacks.c:273:                _PyErr_WriteUnraisableMsg("on converting result "
Modules/_ctypes/callbacks.c-274-                                          "of ctypes callback function",
Modules/_ctypes/callbacks.c-275-                                          callable);

    They can be replaced with PyErr_FormatUnraisable(), but then object will be None. Or we can add new function specially for these cases, which passes both object and a format with arguments for a warning message. I do not know how important to have object.

  9. serhiy-storchaka
    added a commit that references this issue on Nov 2, 2023

    pythongh-108082: Remove _PyErr_WriteUnraisableMsg()

    f7d2530

  10. vstinner commented on Nov 2, 2023

    @vstinner
    vstinner
    on Nov 2, 2023
    MemberAuthor

    They can be replaced with PyErr_FormatUnraisable(), but then object will be None. Or we can add new function specially for these cases, which passes both object and a format with arguments for a warning message. I do not know how important to have object.

    As the author of _PyErr_WriteUnraisableMsg(), I suggest to remove _PyErr_WriteUnraisableMsg() and replace all _PyErr_WriteUnraisableMsg() usage with PyErr_FormatUnraisable().

    My intent for passing object to the hook was to delegate the formatting to give the hook the opportunity to catch exception on formatting the object, and maybe use tracemalloc to see where the object was allocated, etc.

    But I don't think that these features are really useful and we should just skip/remove object, and just pass an error message to the hook.

  11. 2 remaining items

  12. serhiy-storchaka
    closed this as completedon Nov 3, 2023
  13. FullteaR
    added 4 commits that reference this issue on Nov 3, 2023

    pythongh-108082: C API: Add tests for PyErr_WriteUnraisable() (python…

    4071691

    pythongh-108082: Add PyErr_FormatUnraisable() function (pythonGH-111086)

    33462e8

    pythongh-108082: Use PyErr_FormatUnraisable() (pythonGH-111580)

    a6f4f43

    pythongh-108082: Remove _PyErr_WriteUnraisableMsg() (pythonGH-111643)

    8d80f82

  14. serhiy-storchaka commented on Nov 3, 2023

    @serhiy-storchaka
    serhiy-storchaka
    on Nov 3, 2023
    Member

    Now we can replace more PyErr_Clear() with informative PyErr_FormatUnraisable().

  15. aisk
    added 4 commits that reference this issue on Feb 11, 2024

    pythongh-108082: C API: Add tests for PyErr_WriteUnraisable() (python…

    e262ba6

    pythongh-108082: Add PyErr_FormatUnraisable() function (pythonGH-111086)

    7af0c34

    pythongh-108082: Use PyErr_FormatUnraisable() (pythonGH-111580)

    daf14e8

    pythongh-108082: Remove _PyErr_WriteUnraisableMsg() (pythonGH-111643)

    7611523
  16. Glyphack
    added 4 commits that reference this issue on Sep 2, 2024

    pythongh-108082: C API: Add tests for PyErr_WriteUnraisable() (python…

    a87ec4c

    pythongh-108082: Add PyErr_FormatUnraisable() function (pythonGH-111086)

    7a60adc

    pythongh-108082: Use PyErr_FormatUnraisable() (pythonGH-111580)

    56133ef

    pythongh-108082: Remove _PyErr_WriteUnraisableMsg() (pythonGH-111643)

    4f1b225
  17. matthewfeickert
    added a commit that references this issue on Aug 7, 2026

    test: Close file handles in contrib tests for Python 3.14

    0b9e148
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    topic-C-APItype-featureA feature request or enhancement

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions