Repository navigation
[FEA]: Add find_nvidia_dynamic_library to find the DSO location without loading #757
Description
It’d be nice for path finder to gain a capability find_nvidia_dynamic_library, that only returns the DSO location that the path finder would load, but does not actually load it. cuFile and nvJitLink could use this.
Metadata
Metadata
Assignees
Type
Projects
- Status
Activity
rwgk commentedon Jul 15, 2025
I floated this idea a few days ago:
This could be implemented in a few lines by calling
load_nvidia_dynamic_library()in a separate process. This approach guarantees thatload_nvidia_dynamic_library()and find_nvidia_dynamic_library() are 100% consistent. All major parts of the functionality needed exist already (in particular https://github.com/NVIDIA/cuda-python/blob/a93b760e01a16c7f80e7b802095a1d4d069c001e/cuda_bindings/tests/spawned_process_runner.py).kkraus14 commentedon Jul 15, 2025
If a different library loaded in the current process had some RPATH set that influenced where it loaded something from, wouldn't that not be captured in a separate process?
leofang commentedon Jul 15, 2025
Yes I've been thinking about this too. One notable example is how conda Python finds shared libraries via RPATH. nvmath-python currently also uses RPATH. Another potential divergence is if there is another library that already loads what we need, and pathfinder would jus reuse it.
I think
find_nvidia_dynamic_libraryshould present the solution that we would get without being influenced by other DSOs. So in a sense a potential divergence is inevitable.[-]Add `find_nvidia_dynamic_library` to find the DSO location without loading[/-][+][FEA]: Add `find_nvidia_dynamic_library` to find the DSO location without loading[/+]pathfinder: add locate_nvidia_dynamic_lib and find_nvidia_dynamic_lib
srimanachanta commentedon Oct 28, 2026
I opened #3233 to address this. It builds on @rparolin's #1980.
#3233 adds
find_nvidia_dynamic_lib(libname)->strand locate_nvidia_dynamic_lib(libname)->LocatedDynamicLib, matching the existing find_/locate_ pairs for static and bitcode libraries. They return the fileload_nvidia_dynamic_libwould load in a fresh process, without loading it into the caller's process.From #1980 it keeps the
find_nvidia_dynamic_libname, the error contract and the functools.cache on find_. It also keeps the behavior agreed in this thread: libraries already loaded in the current process don't affect the result.The main change is where the search runs. #1980 ran all of
load_nvidia_dynamic_libin a subprocess. #3233 runs the file-based steps (site-packages, conda, CUDA_PATH/CUDA_HOME, Program Files) in the caller's process. It uses the existing canary subprocess only for the native loader search, which can't be observed without loading. A library found in site-packages or conda needs no subprocess. A test checks that for every supported library,locate_nvidia_dynamic_libreturns the same file asload_nvidia_dynamic_librun in a child process.