Repository navigation
Cyclic imports in cuda-bindings causing import errors #789
Description
Is this a duplicate?
- I confirmed there appear to be no duplicate issues for this bug and that I agree to the Code of Conduct
Type of Bug
Compile-time Error
Component
cuda.bindings
Describe the bug
We are using a custom Python environment, where cyclic imports in Python do not work. This is usually not a problem for the vast majority of Python packages, but unfortunately it does cause issues when trying to use cuda-bindings, since there are some cyclic imports between the various cython files.
One example is the cycle between the driver -> _lib.utils -> driver submodules.
Would it be possible to refactor this code to remove the cycle?
How to Reproduce
Run the following code with cuda-bindings installed, which attempts to approximate our import system:
orig_import = __builtins__.__import__
import_stack = []
def import_hook(name, globals=None, locals=None, fromlist=(), *args, **kwargs):
"""Approximate a custom import system that does not allow import cycles."""
stack_entry = (tuple(fromlist) if fromlist is not None else None, name)
if stack_entry in import_stack:
raise ImportError(
f"Import cycle detected: {stack_entry}, stack: {import_stack}"
)
import_stack.append(stack_entry)
res = orig_import(name, globals, locals, fromlist, *args, **kwargs)
import_stack.pop()
return res
__builtins__.__import__ = import_hook
import cuda.bindings.nvvm # works
import cuda.bindings.nvrtc # fails due to driver -> _lib.utils -> driver cycleI get the following error on an Ubuntu system with Python 3.12.9 when running the above code:
Traceback (most recent call last):
File "import_hook_test.py", line 24, in <module>
import cuda.bindings.nvrtc # fails due to driver -> _lib.utils -> driver cycle
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "import_hook_test.py", line 15, in import_hook
res = orig_import(name, globals, locals, fromlist, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "cuda/bindings/nvrtc.pyx", line 1, in init cuda.bindings.nvrtc
File "import_hook_test.py", line 15, in import_hook
res = orig_import(name, globals, locals, fromlist, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "cuda/bindings/driver.pyx", line 1, in init cuda.bindings.driver
File "import_hook_test.py", line 15, in import_hook
res = orig_import(name, globals, locals, fromlist, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "cuda/bindings/_lib/utils.pyx", line 1, in init cuda.bindings._lib.utils
File "import_hook_test.py", line 11, in import_hook
raise ImportError(
ImportError: Import cycle detected: ((), 'cuda.bindings.driver'), stack: [(None, 'cuda.bindings.nvrtc'), ((), 'cuda.bindings.driver'), ((), 'cuda.bindings._lib.utils')]
Expected behavior
Imports with the custom hook without raising an exception.
Operating System
Custom linux-based system
nvidia-smi output
N/A - fails at import
Metadata
Metadata
Assignees
Labels
Type
Projects
- Status
Activity
mdboom commentedon Aug 12, 2025
👋 Hi, I'm new to the project and working on this as a good ramp-up task.
Doing some analysis on this, these are the import cycles created when importing
cuda.bindings.nvrtc:I think I've found a way to remove the cycle involving
cuda.bindings.utils._get_handle(by having modules self-register their types in the handle map, rather than pre-generating it). That gets us something like this (which also imports fewer things by default):The cycle between
cuda.bindings._lib.utilsandcuda.bindings.driveris going to be a lot more challenging. The best I can think of would involve changing howCythonworks to resolve cross-module symbols lazily after import time, but that's "non-trivial".I will probably pursue a patch to remove the
_get_handleloop in any event, since I think it has other advantages.Improve NVIDIA#789: Remove cyclical dependency between {driver|runtim…
gflegar commentedon Aug 18, 2025
👋
Yeah, I was also trying to see how easy it would be to move a few things around as a patch before posting this, and this is exactly where I got stuck and figured out there's a lot of interdependencies between the two files, so moving a few things around is not super easy. (And the error message I see internally is complaining about exactly these two files depending on one another.)
Improve #789: Remove cyclical dependency between cuda.bindings.{drive…
leofang commentedon Aug 23, 2025
@gflegar while we're working on untangling the imports, we'd still like to understand better why this trips you, since the said circular import has been there since the beginning of this project, and it's been working fine over the years and across multiple Python versions. It seems that Cython handles the
cimports properly for regular Python users, and this is different from normal Python imports.@oleksandr-pavlyk and I have looked into if this is related to the multi-phase initialization (I tried rebuilding the package with
DCYTHON_PEP489_MULTI_PHASE_INIT=0but found nothing different). I'd like to know:cuda-bindingsfrom source or pip install the binary wheel?I am looking for ways to reproduce your Python environment without the provided import hook, so as to understand this issue better.
13 remaining items
mdboom commentedon Aug 27, 2025
Progress summary:
3 legitimate cycles have been found when importing all of the top-level modules in
cuda.bindings:I did find a small bug in the reproducer where it wouldn't pop modules off of the stack if they caught and handled and
ImportErrors. The fix is to put thestack.popinfinallyblock. I don't think that makes it no longer match your custom import system (and I confirmed it doesn't seem to hide legitimate cycles), but you may want to confirm on your end.mdboom commentedon Aug 28, 2025
@gflegar: I believe all of the cyclical imports have been resolved at this point. Are you able to test a build from main internally to confirm that we have resolved what you need before we cut a release?
Fix NVIDIA#789: Remove cycle between c.b.cyruntime and c.b._lib.cyrun…
gflegar commentedon Sep 1, 2025
Amazing, thanks!
I was out the last couple of weeks, so I'll try to catch up on the discussion above. I'm on internal integration duty for a different project this week, so I think I'll be able to try out the new version of this early next week and let you know if everything works out.
We haven't used cuda-bindings internally in Google until now, when we started needing it since it's a dependency for CuTeDSL that we want to try out, hence we never discovered this issue.
AFAICT, this is not a cython-specific issue for us, but any Python module that uses circular imports would have the same problem.
I don't know the answers to all of these at the moment. We have a monorepo and a dedicated Python team that maintains a single Python environment for everyone, and it "just works" for the rest of us (until it doesn't, like what happened here 😂 ). The only thing I could find in internal documentation is that circular imports just do not work, and AFAIU this has something to do with the "Hermetic Python" setup that we use (AFAIU whenever we want to build a Python program, the entire Python interpreter and all the dependencies are built from source and packaged up together). I can try finding someone from the Python team to give you a better idea of why this is a problem, but it might take some time.
Neither, our internal runtime environment.
3.12
From source (and we generally do not support pip & binary wheels internally).
3.1.1
Though as mentioned above, the versions themselves probably do not matter, and it's related to the custom runtime environment.
Ah, good catch! Yeah, that's probably correct. I wasn't trying to match the import system with the code, but just generally detect cycles, since I know that is what's not supported by it. I'll check the new version of cuda-bindings next week, and see if there's any issues remaining.
gflegar commentedon Sep 15, 2025
I got to try this now, and got it running. Thanks for all the work!
I still had to apply a small patch to make everything work, due to
cuda.bindings.driverimporting itself through the included_lib/utils.pxifile:Patch
Would this be something we can add upstream as well? I can't reproduce this latest issue with the repro I had above though.
I dug a bit deeper into why we're having these issues in the first place, and it looks like our Python launcher uses a custom module import machinery to allow us to do a few optimizations related to our internal setup. However, this machinery is simpler than the default
importlibimplementation and doesn't have special handling that it has.Specifically, IIUC importlib will do asynchronous module imports, and add modules to
sys.modulesearly, before actually executing the module code (see https://github.com/python/cpython/blob/a68efdf09cfd81c64973d346ed34057f20172543/Lib/importlib/_bootstrap.py#L931)Then, if it encounters a module that is already initializing (e.g., due to cyclic, or self-import), it will block until the code importing the module finished executing (see https://github.com/python/cpython/blob/a68efdf09cfd81c64973d346ed34057f20172543/Lib/importlib/_bootstrap.py#L1377).
Our implementation is completely missing this handling, which I think causes the error.
However, I was still unable to reproduce the latest error in OSS. I tried something like the following, which still succeeded, so I might still be missing something:
kkraus14 commentedon Sep 16, 2025
Reopening the issue based on #789 (comment) and we can close once we've landed the change for
utils.pxiupstreamFix NVIDIA#789: Remove self-cycle in cuda.bindings.driver
mdboom commentedon Sep 16, 2025
@gflegar: Thanks for the report. Your patch makes sense, but I worry that it would add unnecessary additional cache lookups on the critical path. Would you mind trying by PR here to see if that also resolves the issue in your environment?
Fix #789: Remove self-cycle in cuda.bindings.driver (#976)