Repository navigation
Cython cannot use "pycore_frame.h" in Py3.14a4 #130931
Description
Bug report
Bug description:
It's this time of the year again. As discussed before (e.g. #123747), Cython #includes CPython's pycore_frame.h to get access to certain frame features, e.g. their integration into tracebacks.
I noticed that Cython generated code doesn't compile in CPython 3.14 alpha with the following kind of errors:
…\Python\3.14.0-alpha.4\x64\include\internal\pycore_stackref.h(278): error C7555: use of designated initializers requires at least '/std:c++20'
…\Python\3.14.0-alpha.4\x64\include\internal/pycore_frame.h(196): error C7555: use of designated initializers requires at least '/std:c++20'
…\Python\3.14.0-alpha.4\x64\include\internal/pycore_frame.h(196): error C4576: a parenthesized type followed by an initializer list is a non-standard explicit type conversion syntax
…\Python\3.14.0-alpha.4\x64\include\internal/pycore_frame.h(378): error C7555: use of designated initializers requires at least '/std:c++20'
…\Python\3.14.0-alpha.4\x64\include\internal/pycore_frame.h(378): error C4576: a parenthesized type followed by an initializer list is a non-standard explicit type conversion syntax
…\Python\3.14.0-alpha.4\x64\include\internal/pycore_frame.h(379): error C7555: use of designated initializers requires at least '/std:c++20'
…\Python\3.14.0-alpha.4\x64\include\internal/pycore_frame.h(379): error C4576: a parenthesized type followed by an initializer list is a non-standard explicit type conversion syntax
This is from one of our Windows CI runs. Py3.14a4 seems to be what Github Actions currently provides. Cython code expects C99 and something close to (but not necessarily as complete as) C++11 as compiler standards. It targets both C and C++, depending on user needs.
The dependency on pycore_stackref.h was apparently added in #118450:
22b0de2755e [2024-06-27 03:10:43 +0800] GitHub | gh-117139: Convert the evaluation stack to stack refs (#118450)
diff --git a/Include/internal/pycore_frame.h b/Include/internal/pycore_frame.h
index bab92c771a7..1e0368faa5b 100644
--- a/Include/internal/pycore_frame.h
+++ b/Include/internal/pycore_frame.h
@@ -11,6 +11,7 @@ extern "C" {
#include <stdbool.h>
#include <stddef.h> // offsetof()
#include "pycore_code.h" // STATS
+#include "pycore_stackref.h" // _PyStackRef
/* See Objects/frame_layout.md for an explanation of the frame stack
* including explanation of the PyFrameObject and _PyInterpreterFrame
@@ -67,7 +68,7 @@ typedef struct _PyInterpreterFrame {
uint16_t return_offset; /* Only relevant during a function call */
char owner;
/* Locals and stack */
- PyObject *localsplus[1];
+ _PyStackRef localsplus[1];
} _PyInterpreterFrame;
#define _PyInterpreterFrame_LASTI(IF) \What can we do to resolve this?
CPython versions tested on:
3.14
Operating systems tested on:
Windows
Activity
scoder commentedon Mar 7, 2025
@encukou @vstinner
encukou commentedon Mar 7, 2025
Added by @Fidget-Spinner & @colesbury, back in June.
What kind of public/unstable API would Cython need so it could avoid the internal headers?
Fidget-Spinner commentedon Mar 7, 2025
I think we should be able to make it compliant with the C compiler Cython is using
scoder commentedon Mar 7, 2025
Cython directly accesses frames in three places:
We already got rid of direct frame usage for tracing in Py3.13+ by migrating to the new monitoring C-API. Sadly, that currently excludes coverage reporting since
coverage.pycannot use it with plugins yet (such as the Cython coverage plugin). That'll eventually come, I hope, but it's not a CPython issue. It just means that you cannot currently use coverage reporting of Cython code in Python 3.14, as long ascoverage.pyneeds this (old, pre-monitoring) code that requirespycore_frame.h(which doesn't currently compile in C++11):https://github.com/cython/cython/blob/f193ba860b94a7139cffd8b0d11b7ef977aa10d3/Cython/Utility/Profile.c#L629-L639
For exceptions, all we'd need (AFAICT) is a
PyFrame_SetLineNumber(frame, lineno). This has been mentioned before in #118720 but was never implemented.The coroutine support then needs to inject frames into the current stack. For this, we currently read and write
f_back:https://github.com/cython/cython/blob/f193ba860b94a7139cffd8b0d11b7ef977aa10d3/Cython/Utility/Coroutine.c#L784-L831
https://github.com/cython/cython/blob/f193ba860b94a7139cffd8b0d11b7ef977aa10d3/Cython/Utility/Coroutine.c#L870-L897
There was a Cython ticket about this created by @markshannon a while ago that also never got anywhere.
cython/cython#4484
Fidget-Spinner commentedon Mar 7, 2025
@scoder could you please point out the C++ standard violations in your CPython alpha? The line number you posted for the alpha doesn't seem to match up with the current source. I think there's been some changes between then. Thanks!
scoder commentedon Mar 7, 2025
Yeah, this output was copied from a Github Actions run of MSVC, which notoriously reports randomly displaced line numbers. I regularly end up looking broadly around the reported lines to guess my way towards potential code candidates for the reported errors and warnings. I have no idea how people can make productive use of such a compiler.
I actually failed to find any designated initializers in
pycore_frame.hmyself, but they are definitely used inpycore_stackref.h. The issue then is that they long existed in C99 but only came into C++ very late, in C++20. Usually not an issue for internal headers in a C project like CPython these days, unless someone needs to access internal headers from C++ code.Fidget-Spinner commentedon Mar 7, 2025
Ok if it's just removing the designated initializers that should be easy enough to do. I'll take this up.
encukou commentedon Mar 7, 2025
I'm afraid the compiler complaints are about two features: C7555 is for designated initializers but C4576 is for compound literals (the cast-like syntax).
Fidget-Spinner commentedon Mar 7, 2025
I think I'm going to wait for @markshannon to land his PR on stackrefs first.
Fidget-Spinner commentedon Mar 7, 2025
@encukou the cast-like syntax is going to be hard to fix without losing some performance on msvc. I'm not sure what to do for those. Any thoughts?
encukou commentedon Mar 7, 2025
Expose the frame struct (and only that) more? Unstable API is probably the best place for it given Cython's usage; unfortunately #31530 predated PEP-689.
So, rename
struct _frametoPyUnstable_FrameObject, move it toInclude/cpython/frameobject.hand document it?encukou commentedon Mar 7, 2025
(I'm happy to do that, especially the docs part, if that seems right to y'all)
39 remaining items
encukou commentedon Apr 29, 2025
To set expectations: I won't work on this in the foreseeable future. Happy to do reviews.
vstinner commentedon Oct 9, 2025
The initial issue (Cython cannot use "pycore_frame.h" in Py3.14a4) has been fixed. Can we close this issue?
picnixz commentedon Oct 9, 2025
I believe so (sorry for all those labels; I thought it was a new issue)
encukou commentedon Oct 9, 2025
I'd prefer keeping this open for #130931 (comment)
picnixz commentedon Oct 9, 2025
Couldn't we open a separate issue for this one in particular? it's burried in the list of comments and not visible by default. It's also a separate matter (a new feature that will need docs & co, nothing to do with the current title)