Repository navigation
Inconsistency in Cython object handling #143908
Description
Bug report
Bug description:
There seems to be no consensus in the codebase on how to handle Cython objects. Both inspect.iscoroutinefunction() and inspect.iscoroutine() are defined in the documentation for async def functions and their objects, respectively. Therefore, the following code should be valid for them:
#!/usr/bin/env python3
import inspect
async def coroutine_function() -> None:
pass
def main() -> None:
print(inspect.iscoroutinefunction(coroutine_function)) # `True`
print(inspect.iscoroutine(coro := coroutine_function())) # `True`
coro.close() # to avoid `RuntimeWarning`
if __name__ == "__main__":
main()The same semantics are also expected by typeshed, which defines annotations as follows (and thus declares support for introspection):
...
@overload
def iscoroutinefunction(obj: object) -> TypeGuard[Callable[..., CoroutineType[Any, Any, Any]]]: ...
...
def iscoroutine(object: object) -> TypeIs[CoroutineType[Any, Any, Any]]: ...
...However, after cythonize -i -3, the code changes its behavior:
True # `inspect.iscoroutinefunction()`
False # `inspect.iscoroutine()`This is because Cython uses its own objects for compiled functions and generators/coroutines/asynchronous generators. inspect.is<object>() returns True only for uncompiled objects, as it relies on the types module, but inspect.is<object>function() relies on checking function flags and handles function-like objects in a special way (for compatibility with Cython). As a result, we get the contradictory "coroutine function does not return a coroutine".
The problem is even deeper than one might imagine:
- Cython objects do not actually correspond completely to their counterparts from CPython. See [BUG]
inspect.get*state()does not work for compiled objects on Python ≥3.11 cython/cython#7448. - Mypy avoids using the types module to infer the return type due to possible incompatibility with Cython. See mypy should use CoroutineType instead of Coroutine for return types of async def functions mypy#18635 and Allow
types.GeneratorTypeto be used instead ofcollections.abc.Generatoras the return type of generator functions mypy#20522. - If the user wants to handle objects regardless of whether the module has been compiled or not, they have to come up with their own way of checking objects. See Add a
collection.abc.FunctionABC for function-like objects #131983.
What if we want to define a universal decorator that works for both regular and asynchronous functions? We could use inspect.iscoroutinefunction(), but it has very limited detection (in particular, it cannot detect object with async def __call__()), and library developers will only be able to rely on inspect.markcoroutinefunction() after the Python 3.11's EOF. A frequently suggested alternative is to check the return value's type using inspect.isawaitable(), but this is actually shooting oneself in the foot, as it will lead to false positives for future-like objects (such as asyncio future objects, which are also awaitable). The most logical solution would be to use inspect.iscoroutine(), but it does not support Cython (and in fact, it is even less useful; see GrahamDumpleton/wrapt#236 (comment)).
It seems to me that the question of whether the standard library (and related tools) should be Cython-aware has not been fully explored, which leads to such contradictions. I marked the issue as a bug, since it violates the expected behavior for any consistent interpretation.
CPython versions tested on:
3.9, 3.10, 3.11, 3.12, 3.13, 3.14
Operating systems tested on:
Linux
Activity
ZeroIntensity commentedon Jan 16, 2026
Why is this necessarily a bad thing? If all you want to do is
awaitsomething, I don't see why a future would be bad.Anyways, the ideal solution is to avoid using
inspectentirely.inspect.iscoroutineis only meant to detect native coroutines (types.CoroutineTypeinstances), not other coroutine types. If a library wants to support compiled coroutines from Cython or something like that, then they should be usingisinstanceoncollections.abc.Coroutine.I don't think CPython should add special cases solely for Cython. That would lead to an increase in maintenance burden when Cython inevitably changes how they implement coroutines or similar, and we don't have a great way to test it without adding Cython to the test suite.
If Cython wants new generic APIs for coroutines, that seems feasible, but I don't see any such proposal here.
x42005e1f commentedon Jan 16, 2026
Why should a user library care about Cython support when it works with Python-level code? Any Python code can be compiled via
cythonize. The fact that Cython objects do not inherit fromtypes.<Object>Type(while Nuitka does) is an intentional design decision, and both CPython and Mypy already take care of this for some reason (why?).But it already has them, does not it?
x42005e1f commentedon Jan 16, 2026
To be clear, I am not proposing anything in this issue. Neither to strengthen support for Cython nor to remove it completely. I am merely summarizing that, because the question of Cython support has apparently never been seriously discussed, the implementation contains conflicting decisions, and this uncertainty significantly complicates typing and introspection (and violates the expected behavior).
Mypy avoids using
types.<Object>Typebecause they are not compatible with Cython. typeshed usestypes.<Object>Typein standard library annotations, which is not compatible with Cython. It seems like the same Python, but they contradict each other. And the same thing happens in CPython itself.ZeroIntensity commentedon Jan 16, 2026
I don't know, but this is a better question for the Cython people. If they want a better API, then we can consider adding something to CPython.
None of these comments are actually special to Cython. They're merely generalizations, and Cython is an example of where they're hit.
This sounds like something that should be a DPO thread, then. Our issue tracker isn't a good place for discussion/summaries. If there's problems with typeshed's annotations, then go report it over there.
x42005e1f commentedon Jan 16, 2026
I think we have strayed from the point. Why does
iscoroutinefunction()have a generalized implementation, butiscoroutine()does not? The former could rely onisfunction(), but it does not. At the same time, both are defined forasync deffunctions (see the documentation). Whether we are considering Cython here or not is secondary.I posted this as an issue because I considered the observed behavior to be inconsistent. I used the annotation problem as an example (after all, type checkers reflect runtime behavior). That is a consequence, not a cause.
I apologize for not being able to convey my thoughts to you clearly. If you believe that the issue should be closed, that is fine.
x42005e1f commentedon Jan 16, 2026
And in general, what do you consider "special to Cython"? In the pursuit of generalization, almost any case can be generalized to "object-like" things. That is how #68504 was resolved. People will avoid using implementation-specific names, attributes, and types, but the intention remains the same.
ZeroIntensity commentedon Jan 16, 2026
"generalized" is doing some heavy lifting here.
iscoroutinefunctionis basically defined as "if I call this object, will the return value be awaitable?" whereasiscoroutineis defined as "is this object a native coroutine?"In hindsight,
iscoroutineshould have been namedisnativecoroutine(or perhaps not added at all and send users toisinstanceinstead), but that ship has sailed.My concern is that there's not really anything that is actionable here. It's fine to discuss whether something is a bug, but that should happen on DPO so it gets more eyes. Issues fly too low under the radar.
I meant "special to Cython" as in implementing fixes that are only applicable to Cython. If something is a problem across multiple C coroutine implementations, then that's probably worth fixing because that would be indicative of a more general problem regarding non-native coroutines. But, as far as I can tell, this issue is more about "how should we handle Cython coroutines?" The answer is we shouldn't; we should handle non-native coroutines where we can, with Cython as an example use case.
I'm going to close this on the basis that it will likely just collect dust on the issue tracker, and it doesn't look like there's any actionable items here anyway. Please create a thread on the "Core Development" category of DPO so we can get some more opinions. (For that thread though, please focus the question more on "how do we support other types of coroutines?" rather than "how do we support Cython?")
refactor!: avoid native types