Windows: CUDA never initializes, NVRTC is looked up as libnvrtc.so #35

What happens

On Windows the CUDA backend never initializes. Every GPU command falls back to Vulkan and prints:

CUDA unavailable (CUDA init: the NVRTC library (libnvrtc.so) is not installed; sm_70 needs the CUDA 12.x toolkit/NVRTC (13.x dropped Volta support)), using Vulkan

--backend cuda makes it a hard error instead of a fallback, so there is no way to use CUDA on Windows in this build.

Environment

• RustingStrata v1.2.2 (release binary rustingstrata-windows-x86_64.exe, Windows 11 x64)
• RTX 3060 12 GB, driver 616.56 (no CUDA Toolkit installed, only the driver)
• Model: Qwen3.6-35B-A3B Q4_K_M GGUF (20.3 GB)

Why I think this is the library name, not a missing install

NVRTC is available and loads fine on this machine:

• pip install --target ... nvidia-cuda-nvrtc-cu12 provides nvrtc64_120_0.dll + nvrtc-builtins64_129.dll
• LoadLibrary("E:\\.rustingstrata\\libnvrtc.so") returns a valid handle (the file is a copy of nvrtc64_120_0.dll), so the DLL and its dependencies load
• strings rustingstrata.exe contains only the Linux soname; there is no nvrtc64_*.dll name anywhere in the binary:

  $ rg -a -o "nvrtc[0-9A-Za-z_.\\-]{0,20}" rustingstrata.exe | sort -u
  nvrtc
  nvrtc.so
  ...
  libnvrtc.so
  

That looks like a hardcoded libnvrtc.so in the loader path, which Windows will not resolve to nvrtc64_120_0.dll.

Things I tried (all fail identically)

1. nvrtc64_120_0.dll and nvrtc-builtins64_129.dll copied next to rustingstrata.exe
2. Same DLL copied as libnvrtc.so (and as libnvrtc.so.dll in case the loader appends an extension)
3. bin/, lib/, lib64/ subdirectories next to the exe with libnvrtc.so
4. Adding the exe directory to PATH, and setting CUDA_PATH to the directory that contains the DLL
5. Full CUDA Toolkit is not installed, so this was tested with the redistributable DLLs from PyPI (12.9, ABI name nvrtc64_120_0.dll)

Expected

On Windows, the runtime should probe the Windows names (nvrtc64_120_0.dll, nvrtc64_112_0.dll, or the DLLs shipped by the toolkit) and report a clear error only when none of them load. Ideally it also respects CUDA_PATH.

Workaround

--backend vulkan works well on the same machine: 60.7 tok/s decode / 248 tok/s prefill at 512-token prompts, 53.4 / 808 at 4K (Qwen3.6-35B-A3B Q4_K_M, 32K context). So this is only about the CUDA path.

Happy to test a patched build or provide the exact LoadLibrary traces.