Skip to content

fix(cuda.core): report the exporter's NUMA node on imported pinned pools - #3530

Draft
juenglin wants to merge 1 commit into
NVIDIA:mainfrom
juenglin:fix-3508-pinned-ipc-numa-id
Draft

juenglin wants to merge 1 commit into
NVIDIA:mainfrom
juenglin:fix-3508-pinned-ipc-numa-id

Conversation

@juenglin

@juenglin juenglin commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Description

closes #3508

An imported PinnedMemoryResource reported numa_id == 0 when the driver cannot report the pool's location (a CUDA 12 build or a driver before 13.2); #3516 fixed the case where it can. Now the exporter's node travels with the pickle, and PinnedMemoryResource.from_allocation_handle takes an optional numa_id for raw handles. An import whose node is still unknown reports 0 as before and sends no node when re-pickled. A negative numa_id, or one that differs from the pool's known node, raises ValueError.

Tested with a CUDA 13.5 build; the no-location-query paths are covered by patching driver_version in the child process. A CUDA 12 build and a multi-node NUMA host were not tested.

Checklist

  • I have read the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

An imported PinnedMemoryResource reported numa_id 0 whenever the driver
could not report the pool's location (a CUDA 12 build or a driver before
13.2). Unpickling a resource now carries the exporter's node, and
PinnedMemoryResource.from_allocation_handle accepts an optional numa_id
for a raw allocation handle. An import whose node is unknown still reports
0 and does not pass it on when pickled.

from_allocation_handle raises ValueError for a negative numa_id, and for
one that differs from the pool's known node, whether the driver reported
it or the resource was already registered in the process.

Closes NVIDIA#3508.

Signed-off-by: Ralf Juengling <rjuengling@nvidia.com>
@juenglin juenglin added this to the cuda.core 1.4.0 milestone Nov 21, 2026
@juenglin juenglin added bug Something isn't working cuda.core Everything related to the cuda.core module labels Nov 21, 2026
@copy-pr-bot

copy-pr-bot Bot commented Nov 21, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@juenglin juenglin self-assigned this Nov 21, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working cuda.core Everything related to the cuda.core module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]: PinnedMemoryResource imported over IPC reports numa_id 0 instead of the exporter's node

1 participant