Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions Doc/c-api/exceptions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,44 @@ For convenience, some of these functions will always return a

.. versionadded:: 3.5

.. c:function:: void PyErr_ChainExceptions(PyObject *exc, PyObject *val, PyObject *tb)

Set the provided exception as the context for the current exception. If there
is no exception currently set (:c:func:`PyErr_Occurred` returns ``0``) this
function behaves like :c:func:`PyErr_Restore`, but if an exception is
already set, set the provided exception as the context (accessible from
Python through the :attr:`__context__` attribute of the exception) of the
one that is currently set.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it the other way around - the existing exception becomes the context of the provided exception?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code calls PyException_SetContext(val2, val); where val is the provided one and val2 is the one that is currently set. PyException_SetContext sets the second argument as the context for the first.

So unless I am missing something (is late here 😅 ) this means that "The provided exception (val) becomes the context of the existing exception (val2)"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I was thinking we are raising the provided exception in the context of the existing one, but that's what SetObject is for. Can we think of a more descriptive name for this function? I confused the direction, and you confused cause/context because "chain" is ambiguous here.

How about PyErr_RestoreContext? IIUC this describes what you use this function for - putting back a fetched exception, as the context of any exception that happened between the fetch and now. (Should something like the previous sentence be in the doc?)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or possibly PyErr_RestoreAsContext?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general authors look for "chain" in the docs because that's the action they want to do, so I would really recommend to have "chain" in the name.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That may be the case, but I think we should promote clear language, and "chain" in this context is vague. The doc can contain "chain" in the prose for the benefit of people who search for it (for now). But should it be part of the function name for all eternity?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is unclear to me. I agree with the ambiguity but what users if the function want is a higher level approach to the python behaviour: an exception is raised when another exception was there and I want to do in C the same thing as Python does.

I'm not a native speaker do I trust your judgement regarding how ambiguous this sounds and that we need a better name so let's search for one.

Maybe we should make a section in the docs about "exception chaining" and offer two APIs: one for cause and another for context

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iritkatriel Maybe you can suggest a more specific name that still retains "chain" as part of it?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The context is an implicitly chained exception (the cause is explicitly chained). There is no need to have both chained and context in the name. If we have only one of them I think it should be context and not chain.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the equivalence to PyErr_Restore when no exception is currently set, my public naming suggestion would be PyErr_RestoreOrChain, with the shorthand documentation being "Restores the given exception state, or chains it as the exception context if an exception is already set".

The key point to emphasise is that the function takes the same arguments as PyErr_Restore (retrieved from a prior PyErr_Fetch call), and generally does the same thing, but handles currently set exceptions differently: instead of clearing them, it leaves them in place, and "restores" the given exception as the exception context, rather than as the active exception.


The exception type should be a class. Do not pass an invalid exception type
or value. This function steals a reference to each of its arguments in the

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could it not steal its arguments?
Functions that don't follow the usual calling convention are harder to reason about; it would be good to treat them as historical exceptions.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It steals the arguments because of what it calls. It had the same calling convention as Pyerr_Restore.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, stealing arguments is not an uncommon calling convention.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stealing references is only common because much of the public API was born as internal APIs, and we often try to avoid redundant refcount adjustments in internal APIs.

It took us a long time to realise that stealing references in public APIs makes things confusing for API clients, and can easily lead to double decref bugs (which are more potentially destructive than the object leaks that can arise from missing decref bugs, since deleting objects early can lead to access violations). So I think the new public API should be a wrapper around the internal API that avoids stealing references.

I changed my mind on this point, as PyErr_Fetch and PyErr_Restore are a matched pair. If we go with my PyErr_RestoreOrChain naming suggestion, then the refcounting semantics of the new API should match those of PyErr_Restore (which is to take back the references that PyErr_Fetch created)

same fashion as :c:func:`PyErr_Restore` does.

.. warning::

The caller is responsible for ensuring that this call won't create
any cycles in the exception context chain.

@iritkatriel iritkatriel Aug 17, 2021 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the doc should say what happens if you violate the rule. I think in reality it will just create the cycle and if it doesn't bother you then good for you.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the doc should say what happens if you violate the rule.

Is undefined behavior, technically as the contract of this function withing CPython is that cycles should not be created. Some parts of the exception machinery can break (or at least they used to) in presence of cycles.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But when the exception machinery breaks in the presence of a cycle we consider that to be a bug in python.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is the caller supposed to ensure this? Is there any API for checking it?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no API to check this, but to create a cycle with exceptions it needs to be s deliberate action. There is not an easy way to accidentally crate an exception cycle

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But when the exception machinery breaks in the presence of a cycle we consider that to be a bug in python.

I think the movement is in the direction of trying to prevent / avoid the creation of cycles in the first place (as was mentioned in the discussion for this recent ticket, for example). The warning being added here is in line with that.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per my comment below, the easiest way to violate this requirement is to call the API without setting a new exception first (or clearing the existing one). Buggy code (using my API naming suggestion from above):

PyErr_Fetch(&type, &value, &traceback);
// Exception state is not cleared, no new exception is set
PyErr_RestoreOrChain(type, value, traceback); // Sets exception as its own context

Because the exception might not have been instantiated yet when PyErr_Fetch() is called, there's no easy way for the API to reliably detect this error, so the onus is on the caller to manage the exception state correctly.


See :ref:`tut-exception-chaining` for more information regarding the
semantics of exception chaining.

Example usage:

.. code-block:: c

PyObject *type, *value, *traceback;

/* Fetch the original exception */
PyErr_Fetch(&type, &value, &traceback);

/* ... code that might produce other errors ... */

/* Set the original exception as the context for the new exception, and set the new exception as the current exception */
PyErr_ChainExceptions(type, value, traceback);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This example is misleading, as if the code doesn't set a new exception, the fetched exception will end up being chained with itself instead of being restored. To safely invoke the new API unconditionally, the code needs to look like this:

    /* Fetch and clear the original exception */
    PyErr_Fetch(&type, &value, &traceback);
    PyErr_Clear();

    /* ... code that might produce other errors ... */

    /* Either restore the original exception, or set it as the context for any new exception */
    PyErr_RestoreOrChain(type, value, traceback);

Skipping the PyErr_Clear() call is only OK if the code definitely sets a new active exception.



.. versionadded:: 3.11


.. c:function:: void PyErr_SetNone(PyObject *type)

Expand Down
1 change: 1 addition & 0 deletions Doc/data/stable_abi.dat
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@ var,PyEllipsis_Type,3.2,
var,PyEnum_Type,3.2,
function,PyErr_BadArgument,3.2,
function,PyErr_BadInternalCall,3.2,
function,PyErr_ChainExceptions,3.11,
function,PyErr_CheckSignals,3.2,
function,PyErr_Clear,3.2,
function,PyErr_Display,3.2,
Expand Down
5 changes: 5 additions & 0 deletions Doc/whatsnew/3.11.rst
Original file line number Diff line number Diff line change
Expand Up @@ -295,6 +295,11 @@ Changes in the Python API

C API Changes
=============

* Add a new :c:func:`PyErr_ChainExceptions` function that allows to chain exceptions
using the C-API. See :ref:`tut-exception-chaining` for more information
regarding the semantics of exception chaining. (Contributed by Pablo Galindo in :issue:`44938`.)

* Add a new :c:func:`PyType_GetName` function to get type's short name.
(Contributed by Hai Shi in :issue:`42035`.)

Expand Down
2 changes: 0 additions & 2 deletions Include/cpython/pyerrors.h
Original file line number Diff line number Diff line change
Expand Up @@ -89,8 +89,6 @@ PyAPI_FUNC(void) _PyErr_GetExcInfo(PyThreadState *, PyObject **, PyObject **, Py

/* Context manipulation (PEP 3134) */

PyAPI_FUNC(void) _PyErr_ChainExceptions(PyObject *, PyObject *, PyObject *);

/* Like PyErr_Format(), but saves current exception as __context__ and
__cause__.
*/
Expand Down
3 changes: 3 additions & 0 deletions Include/pyerrors.h
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ PyAPI_FUNC(void) PyErr_Restore(PyObject *, PyObject *, PyObject *);
PyAPI_FUNC(void) PyErr_GetExcInfo(PyObject **, PyObject **, PyObject **);
PyAPI_FUNC(void) PyErr_SetExcInfo(PyObject *, PyObject *, PyObject *);
#endif
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030B0000
PyAPI_FUNC(void) PyErr_ChainExceptions(PyObject *, PyObject *, PyObject *);
#endif

/* Defined in Python/pylifecycle.c

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Expose :c:func:`PyErr_ChainExceptions` in the stable API. Patch by Pablo
Galindo.
2 changes: 2 additions & 0 deletions Misc/stable_abi.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2149,6 +2149,8 @@ function PyType_GetName
added 3.11
function PyType_GetQualName
added 3.11
function PyErr_ChainExceptions
added 3.11

# (Detailed comments aren't really needed for further entries: from here on
# we can use version control logs.)
2 changes: 1 addition & 1 deletion Modules/_asynciomodule.c
Original file line number Diff line number Diff line change
Expand Up @@ -2955,7 +2955,7 @@ task_step(TaskObj *task, PyObject *exc)
PyObject *et, *ev, *tb;
PyErr_Fetch(&et, &ev, &tb);
leave_task(task->task_loop, (PyObject*)task);
_PyErr_ChainExceptions(et, ev, tb);
PyErr_ChainExceptions(et, ev, tb);
return NULL;
}
else {
Expand Down
2 changes: 1 addition & 1 deletion Modules/_io/_iomodule.c
Original file line number Diff line number Diff line change
Expand Up @@ -497,7 +497,7 @@ _io_open_impl(PyObject *module, PyObject *file, const char *mode,
PyObject *exc, *val, *tb, *close_result;
PyErr_Fetch(&exc, &val, &tb);
close_result = _PyObject_CallMethodIdNoArgs(result, &PyId_close);
_PyErr_ChainExceptions(exc, val, tb);
PyErr_ChainExceptions(exc, val, tb);
Py_XDECREF(close_result);
Py_DECREF(result);
}
Expand Down
4 changes: 2 additions & 2 deletions Modules/_io/bufferedio.c
Original file line number Diff line number Diff line change
Expand Up @@ -526,7 +526,7 @@ buffered_close(buffered *self, PyObject *args)
}

if (exc != NULL) {
_PyErr_ChainExceptions(exc, val, tb);
PyErr_ChainExceptions(exc, val, tb);
Py_CLEAR(res);
}

Expand Down Expand Up @@ -2239,7 +2239,7 @@ bufferedrwpair_close(rwpair *self, PyObject *Py_UNUSED(ignored))
Py_DECREF(ret);
ret = _forward_call(self->reader, &PyId_close, NULL);
if (exc != NULL) {
_PyErr_ChainExceptions(exc, val, tb);
PyErr_ChainExceptions(exc, val, tb);
Py_CLEAR(ret);
}
return ret;
Expand Down
2 changes: 1 addition & 1 deletion Modules/_io/fileio.c
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ _io_FileIO_close_impl(fileio *self)
}
rc = internal_close(self);
if (res == NULL)
_PyErr_ChainExceptions(exc, val, tb);
PyErr_ChainExceptions(exc, val, tb);
if (rc < 0)
Py_CLEAR(res);
return res;
Expand Down
2 changes: 1 addition & 1 deletion Modules/_io/iobase.c
Original file line number Diff line number Diff line change
Expand Up @@ -240,7 +240,7 @@ _io__IOBase_close_impl(PyObject *self)

PyErr_Fetch(&exc, &val, &tb);
rc = _PyObject_SetAttrId(self, &PyId___IOBase_closed, Py_True);
_PyErr_ChainExceptions(exc, val, tb);
PyErr_ChainExceptions(exc, val, tb);
if (rc < 0) {
Py_CLEAR(res);
}
Expand Down
4 changes: 2 additions & 2 deletions Modules/_io/textio.c
Original file line number Diff line number Diff line change
Expand Up @@ -2863,7 +2863,7 @@ _io_TextIOWrapper_tell_impl(textio *self)
PyObject *type, *value, *traceback;
PyErr_Fetch(&type, &value, &traceback);
res = _PyObject_CallMethodIdOneArg(self->decoder, &PyId_setstate, saved_state);
_PyErr_ChainExceptions(type, value, traceback);
PyErr_ChainExceptions(type, value, traceback);
Py_DECREF(saved_state);
Py_XDECREF(res);
}
Expand Down Expand Up @@ -3078,7 +3078,7 @@ _io_TextIOWrapper_close_impl(textio *self)

res = _PyObject_CallMethodIdNoArgs(self->buffer, &PyId_close);
if (exc != NULL) {
_PyErr_ChainExceptions(exc, val, tb);
PyErr_ChainExceptions(exc, val, tb);
Py_CLEAR(res);
}
return res;
Expand Down
2 changes: 1 addition & 1 deletion Modules/_io/winconsoleio.c
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ _io__WindowsConsoleIO_close_impl(winconsoleio *self)
PyErr_Fetch(&exc, &val, &tb);
rc = internal_close(self);
if (res == NULL)
_PyErr_ChainExceptions(exc, val, tb);
PyErr_ChainExceptions(exc, val, tb);
if (rc < 0)
Py_CLEAR(res);
return res;
Expand Down
2 changes: 1 addition & 1 deletion Modules/_ssl.c
Original file line number Diff line number Diff line change
Expand Up @@ -574,7 +574,7 @@ PySSL_ChainExceptions(PySSLSocket *sslsock) {
if (sslsock->exc_type == NULL)
return 0;

_PyErr_ChainExceptions(sslsock->exc_type, sslsock->exc_value, sslsock->exc_tb);
PyErr_ChainExceptions(sslsock->exc_type, sslsock->exc_value, sslsock->exc_tb);
sslsock->exc_type = NULL;
sslsock->exc_value = NULL;
sslsock->exc_tb = NULL;
Expand Down
2 changes: 1 addition & 1 deletion Modules/_zoneinfo.c
Original file line number Diff line number Diff line change
Expand Up @@ -228,7 +228,7 @@ zoneinfo_new_instance(PyTypeObject *type, PyObject *key)
PyObject *exc, *val, *tb;
PyErr_Fetch(&exc, &val, &tb);
PyObject *tmp = PyObject_CallMethod(file_obj, "close", NULL);
_PyErr_ChainExceptions(exc, val, tb);
PyErr_ChainExceptions(exc, val, tb);
if (tmp == NULL) {
Py_CLEAR(self);
}
Expand Down
2 changes: 1 addition & 1 deletion Objects/odictobject.c
Original file line number Diff line number Diff line change
Expand Up @@ -1617,7 +1617,7 @@ _PyODict_SetItem_KnownHash(PyObject *od, PyObject *key, PyObject *value,
PyObject *exc, *val, *tb;
PyErr_Fetch(&exc, &val, &tb);
(void) _PyDict_DelItem_KnownHash(od, key, hash);
_PyErr_ChainExceptions(exc, val, tb);
PyErr_ChainExceptions(exc, val, tb);
}
}
return res;
Expand Down
2 changes: 1 addition & 1 deletion Objects/weakrefobject.c
Original file line number Diff line number Diff line change
Expand Up @@ -1006,7 +1006,7 @@ PyObject_ClearWeakRefs(PyObject *object)

tuple = PyTuple_New(count * 2);
if (tuple == NULL) {
_PyErr_ChainExceptions(err_type, err_value, err_tb);
PyErr_ChainExceptions(err_type, err_value, err_tb);
return;
}

Expand Down
1 change: 1 addition & 0 deletions PC/python3dll.c
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,7 @@ EXPORT_FUNC(PyDict_Values)
EXPORT_FUNC(PyDictProxy_New)
EXPORT_FUNC(PyErr_BadArgument)
EXPORT_FUNC(PyErr_BadInternalCall)
EXPORT_FUNC(PyErr_ChainExceptions)
EXPORT_FUNC(PyErr_CheckSignals)
EXPORT_FUNC(PyErr_Clear)
EXPORT_FUNC(PyErr_Display)
Expand Down
2 changes: 1 addition & 1 deletion Python/ceval.c
Original file line number Diff line number Diff line change
Expand Up @@ -697,7 +697,7 @@ _Py_FinishPendingCalls(PyThreadState *tstate)
PyObject *exc, *val, *tb;
_PyErr_Fetch(tstate, &exc, &val, &tb);
PyErr_BadInternalCall();
_PyErr_ChainExceptions(exc, val, tb);
PyErr_ChainExceptions(exc, val, tb);
_PyErr_Print(tstate);
}
}
Expand Down
4 changes: 2 additions & 2 deletions Python/errors.c
Original file line number Diff line number Diff line change
Expand Up @@ -510,7 +510,7 @@ PyErr_SetExcInfo(PyObject *p_type, PyObject *p_value, PyObject *p_traceback)
The caller is responsible for ensuring that this call won't create
any cycles in the exception context chain. */
void
_PyErr_ChainExceptions(PyObject *exc, PyObject *val, PyObject *tb)
PyErr_ChainExceptions(PyObject *exc, PyObject *val, PyObject *tb)
{
if (exc == NULL)
return;
Expand All @@ -519,7 +519,7 @@ _PyErr_ChainExceptions(PyObject *exc, PyObject *val, PyObject *tb)

if (!PyExceptionClass_Check(exc)) {
_PyErr_Format(tstate, PyExc_SystemError,
"_PyErr_ChainExceptions: "
"PyErr_ChainExceptions: "
"exception %R is not a BaseException subclass",
exc);
return;
Expand Down
2 changes: 1 addition & 1 deletion Python/import.c
Original file line number Diff line number Diff line change
Expand Up @@ -655,7 +655,7 @@ remove_module(PyThreadState *tstate, PyObject *name)
}
}

_PyErr_ChainExceptions(type, value, traceback);
PyErr_ChainExceptions(type, value, traceback);
}


Expand Down
4 changes: 2 additions & 2 deletions Python/traceback.c
Original file line number Diff line number Diff line change
Expand Up @@ -252,7 +252,7 @@ PyTraceBack_Here(PyFrameObject *frame)
PyErr_Fetch(&exc, &val, &tb);
newtb = _PyTraceBack_FromFrame(tb, frame);
if (newtb == NULL) {
_PyErr_ChainExceptions(exc, val, tb);
PyErr_ChainExceptions(exc, val, tb);
return -1;
}
PyErr_Restore(exc, val, newtb);
Expand Down Expand Up @@ -294,7 +294,7 @@ void _PyTraceback_Add(const char *funcname, const char *filename, int lineno)
return;

error:
_PyErr_ChainExceptions(exc, val, tb);
PyErr_ChainExceptions(exc, val, tb);
}

static PyObject *
Expand Down