Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions .clang-tidy
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# SPDX-License-Identifier: Apache-2.0

# Layout is clang-format's job (see .clang-format), so clang-tidy does no
# formatting of its own (FormatStyle: none below). Each enabled check is meant
# to be adoptable on its own, so findings can be fixed one check at a time.
#
# Checks are opt-in (-*) by group, minus the ones below. They cannot be
# commented inline because Checks is a single YAML string.
#
# Disabled because they conflict with the C-style CUDA driver API idioms:
# bugprone-easily-swappable-parameters: wrappers of driver calls take many
# adjacent parameters of the same type; the signatures mirror the API.
# bugprone-assignment-in-if-condition: `if (CUDA_SUCCESS != (err = CALL(...)))`
# is the standard error-handling pattern here.
# performance-no-int-to-ptr: opaque handles and file descriptors are
# converted to pointers with reinterpret_cast on purpose.
# modernize-avoid-c-arrays: arrays are passed to and filled by C APIs.
# modernize-return-braced-init-list: `return {};` vs. an explicit type is
# left to the author; it adds noise for the driver-API result structs.
# bugprone-throwing-static-initialization: the handle registries are
# process-lifetime globals whose constructors allocate on some standard
# libraries (MSVC); running out of memory while loading the module is
# fatal either way.
#
# Disabled because they do not match the codebase style:
# performance-enum-size: enum underlying types are left as they are.
# misc-const-correctness: const on every local is not the convention here.
# readability-function-cognitive-complexity: too disruptive.
# readability-identifier-length: short names such as the handle parameter `h`
# are used throughout.
# readability-implicit-bool-conversion: `if (ptr)` and `if (handle)` are the
# idiom for pointers and handles.
# misc-include-cleaner: too noisy with the CUDA and CPython umbrella headers.
# misc-no-recursion: recursion is used deliberately.
# misc-use-anonymous-namespace: file-local functions use `static`.
# modernize-use-trailing-return-type: the code uses `Type\nname()` layout.
Checks: >
-*,
bugprone-*,
-bugprone-easily-swappable-parameters,
-bugprone-assignment-in-if-condition,
-bugprone-throwing-static-initialization,
clang-analyzer-*,
misc-*,
-misc-include-cleaner,
-misc-const-correctness,
-misc-no-recursion,
-misc-use-anonymous-namespace,
modernize-*,
-modernize-use-trailing-return-type,
-modernize-avoid-c-arrays,
-modernize-return-braced-init-list,
performance-*,
-performance-enum-size,
-performance-no-int-to-ptr,
readability-*,
-readability-function-cognitive-complexity,
-readability-identifier-length,
-readability-implicit-bool-conversion
FormatStyle: none
CheckOptions:
# Records whose members are all public (the *Box and guard structs) are plain
# data holders; only a mix of public and private members is flagged.
misc-non-private-member-variables-in-classes.IgnoreClassesWithAllMemberVariablesBeingPublic: true
WarningsAsErrors: '*'
# Only report diagnostics for our own headers, not CUDA/CPython/libstdc++.
HeaderFilterRegex: 'cuda_core/cuda/core/_(cpp|include)/'
ExcludeHeaderFilterRegex: 'cuda_core/cuda/core/_include/(dlpack|aoti_shim)\.h'
8 changes: 8 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,14 @@ repos:

- repo: local
hooks:
# cuda.h and nvrtc.h come from pinned nvidia-cuda-* wheels, so no local CUDA Toolkit is needed.
- id: clang-tidy
name: clang-tidy
entry: python ./toolshed/run_clang_tidy.py --cuda-wheel
language: python
files: ^cuda_core/cuda/core/_cpp/.*\.cpp$

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With files limited to .cpp files under _cpp/, a change to a header (_cpp/rt/*.hpp, _include/), to .clang-tidy, or to toolshed/run_clang_tidy.py skips the hook locally and then fails in CI's --all-files run. Could we widen the pattern to those paths and have the runner analyze every translation unit when it gets a non-.cpp path? A full run is cheap: the pre-commit jobs on this PR, which analyze every translation unit, finish within about a minute of the same jobs on main.

additional_dependencies: [clang-tidy==22.1.8, nvidia-cuda-runtime==13.4.92, nvidia-cuda-nvrtc==13.4.92]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This blocks every Mac developer from committing to the repo, including docs-only and Python-only changes. nvidia-cuda-runtime and nvidia-cuda-nvrtc 13.4.92 publish only manylinux and Windows wheels, with no sdist. pre-commit installs every hook's environment before it filters by file, so on macOS pre-commit run and the commit hook fail for all hooks. CONTRIBUTING says the hooks run on macOS. clang-tidy==22.1.8 has the same problem on Windows-on-ARM: there is no wheel, and the sdist builds LLVM.

Could we add environment markers to these three dependencies, as the config already does for tomli, and have run_clang_tidy.py exit 0 with a notice when the tool, the CUDA headers, or Python.h is missing? CI on Linux and Windows still enforces the check.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. It's a shame there is no way to install the headers only.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This point makes me think (as I contemplated in my OP) that doing this in pre-commit is not the right way to go, since setting up a working development environment across all the configs is so fiddly. I think I'd rather have only one set of build environment setup (the one used for the actual build) and piggy-back on that. Plus, then you are sure you are linting with all of the exact same settings as you are building in. I think it probably makes sense to rework this as a setuptools extension. WDYT, @Andy-Jost?


- id: check-precommit-installed
name: Warn if the pre-commit git hook is not installed
entry: python ./toolshed/check_precommit_installed.py
Expand Down
7 changes: 7 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ Thank you for your interest in contributing to CUDA Python! Based on the type of
- [Review Guidance](#review-guidance)
- [Pull Request Reviews](#pull-request-reviews)
- [Pixi lockfiles](#pixi-lockfiles)
- [Secret Scanning](#secret-scanning)
- [Signing Your Work](#signing-your-work)
- [Code signing](#code-signing)
- [Developer Certificate of Origin (DCO)](#developer-certificate-of-origin-dco)
Expand Down Expand Up @@ -247,6 +248,12 @@ hook environment, so no system-wide installation is required. Vendored code
`.pre-commit-config.yaml`. To format in your editor, install the same version
with `pip install clang-format==23.1.3`.

`cuda_core`'s C++ sources are also checked with `clang-tidy`, configured by
[`.clang-tidy`](.clang-tidy). To apply automatic fixes, run the script directly
with clang-tidy's `--fix` option (without `--cuda-wheel` it uses `CUDA_PATH` or
`CUDA_HOME`) and then run clang-format on the result: `python
toolshed/run_clang_tidy.py --fix <files>`.
Comment on lines +252 to +255

@coderabbitai coderabbitai Bot Oct 27, 2026 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

suggestion: Document the tool needed for direct runs. The pre-commit hook installs clang-tidy in its own environment, but the documented direct command requires clang-tidy on PATH. A contributor who follows only the pre-commit setup gets error: clang-tidy not found on PATH. Add an installation command with the hook’s pinned version before the direct-run command. (pre-commit.com)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. The command also omits --cuda-wheel, so it reads cuda.h from the local toolkit instead of the pinned wheel that the hook uses, and the results can differ.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Andy-Jost suggestion: Document installation of clang-tidy and the CUDA wheel dependencies at the versions pinned in .pre-commit-config.yaml. Then use this command to select wheel headers, as the hook does:

python toolshed/run_clang_tidy.py --cuda-wheel --fix <files>

Keep the local-toolkit command as an explicit alternative, with a note that diagnostics can differ by toolkit version.

You are interacting with an AI system.


## CodeRabbit

This section explains how to use CodeRabbit for CUDA Python pull request review.
Expand Down
4 changes: 2 additions & 2 deletions cuda_core/cuda/core/_cpp/rt/context.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -125,15 +125,15 @@ struct GreenCtxBox {
CUgreenCtx resource;
};

static const ContextBox *
const ContextBox *
get_box(const ContextHandle &h) noexcept
{
const CUcontext *p = h.get();
return reinterpret_cast<const ContextBox *>(reinterpret_cast<const char *>(p) - offsetof(ContextBox, resource));
}

// See REGISTRY_DESIGN.md (Level 1: Driver Handle -> Resource Handle)
static HandleRegistry<CUcontext, ContextHandle> context_registry;
HandleRegistry<CUcontext, ContextHandle> context_registry;

// Create a context handle reference, with optional green context as source.
ContextHandle
Expand Down
2 changes: 1 addition & 1 deletion cuda_core/cuda/core/_cpp/rt/context_scope.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ cleanup_in_context(
}
std::invoke(std::forward<AfterCleanup>(after_cleanup));
if (status != CUDA_SUCCESS || restore != CUDA_SUCCESS) {
char operation_name[160];
char operation_name[kOperationNameSize];
format_operation(operation_name, sizeof(operation_name), name, handle);
if (status != CUDA_SUCCESS) {
report_cuda_error(operation_name, status, detail);
Expand Down
2 changes: 1 addition & 1 deletion cuda_core/cuda/core/_cpp/rt/driver_api.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
namespace cuda_core::rt {

// The pointers. Null until ensure_fn_table() fills the table. See driver_api.hpp.
#define CUDA_CORE_DEFINE_DRIVER_FN(name, introduced) decltype(&name) p_##name = nullptr;
#define CUDA_CORE_DEFINE_DRIVER_FN(name, introduced) decltype(&(name)) p_##name = nullptr;
CUDA_CORE_DRIVER_FUNCTIONS(CUDA_CORE_DEFINE_DRIVER_FN)
#undef CUDA_CORE_DEFINE_DRIVER_FN

Expand Down
4 changes: 2 additions & 2 deletions cuda_core/cuda/core/_cpp/rt/driver_api.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ namespace cuda_core::rt {
X(cuStreamIsCapturing, 10000) \
X(cuThreadExchangeStreamCaptureMode, 10010)

#define CUDA_CORE_DECLARE_DRIVER_FN(name, introduced) extern decltype(&name) p_##name;
#define CUDA_CORE_DECLARE_DRIVER_FN(name, introduced) extern decltype(&(name)) p_##name;
CUDA_CORE_DRIVER_FUNCTIONS(CUDA_CORE_DECLARE_DRIVER_FN)
#undef CUDA_CORE_DECLARE_DRIVER_FN

Expand Down Expand Up @@ -229,7 +229,7 @@ struct Unavailable;
template <class R, class... A>
struct Unavailable<R (*)(A...)> {
static R
call(A...) noexcept
call(A... /*unused*/) noexcept
{
return UnavailableStatus<R>::value;
}
Expand Down
10 changes: 5 additions & 5 deletions cuda_core/cuda/core/_cpp/rt/error.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ namespace {
// Thread-local detail attached to the next raised CUDAError with a matching
// status (see take_last_error_detail()). Written only by propagating helpers.
// The taken copy stays valid until the next take on the same thread.
thread_local char last_error_detail[512] = {0};
thread_local char taken_error_detail[512] = {0};
thread_local char last_error_detail[kErrorMessageSize] = {0};
thread_local char taken_error_detail[kErrorMessageSize] = {0};
thread_local CUresult last_error_detail_status = CUDA_SUCCESS;
} // namespace

Expand Down Expand Up @@ -66,7 +66,7 @@ format_operation(char *buffer, size_t size, const char *operation, unsigned long
void
report_status_code(const char *operation, long code) noexcept
{
char message[256];
char message[kErrorMessageSize];
std::snprintf(message, sizeof(message), "%s failed (status %ld)", operation, code);
report_message(message);
}
Expand All @@ -80,7 +80,7 @@ report_cuda_error(const char *operation, CUresult status, const char *detail) no
if (status == CUDA_SUCCESS || status == CUDA_ERROR_DEINITIALIZED) {
return;
}
char message[512];
char message[kErrorMessageSize];
format_cuda_error(message, sizeof(message), operation, status, detail);
report_message(message);
}
Expand Down Expand Up @@ -122,7 +122,7 @@ note_context_not_restored(CUcontext previous, CUresult operation_status, CUresul
if (DRIVER_CALL(cuCtxGetCurrent, &current) != CUDA_SUCCESS) {
current = nullptr;
}
char cause[128] = {0};
char cause[kErrorMessageSize] = {0};
if (operation_status != CUDA_SUCCESS) {
const char *error_name = nullptr;
if (DRIVER_CALL(cuGetErrorName, restore_status, &error_name) == CUDA_SUCCESS) {
Expand Down
8 changes: 8 additions & 0 deletions cuda_core/cuda/core/_cpp/rt/error.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,17 @@
#pragma once

#include <cuda.h>
#include <cstddef>

namespace cuda_core::rt {

// Sizes of the stack buffers that format error text. Messages are truncated by
// snprintf, never overrun.
constexpr std::size_t kOperationNameSize = 160; // "<call> (handle 0x...)" from format_operation()
constexpr std::size_t kErrorMessageSize = 512; // a full error message or error detail
// A message that embeds one error message after some fixed text.
constexpr std::size_t kNestedMessageSize = kErrorMessageSize + (kErrorMessageSize / 2);

// ============================================================================
// Thread-local error handling
// ============================================================================
Expand Down
65 changes: 39 additions & 26 deletions cuda_core/cuda/core/_cpp/rt/graph.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -83,9 +83,10 @@ struct GraphBox : GraphBoxBase {
GraphAttachmentMap attachments; // Non-owning attachment index.
HandleRegistry<CUgraphNode, GraphNodeHandle> node_handles;

// Not noexcept: the std::map and HandleRegistry members allocate.
GraphBox(
CUgraph resource_, GraphHierarchy *hierarchy_, GraphBox *parent_ = nullptr, CUgraphNode owner_node_ = nullptr
) noexcept
)
Comment thread
coderabbitai[bot] marked this conversation as resolved.
: GraphBoxBase{resource_}, hierarchy(hierarchy_), parent(parent_), owner_node(owner_node_)
{
}
Expand All @@ -106,7 +107,7 @@ struct GraphHierarchy {

// See REGISTRY_DESIGN.md (Level 1: Driver Handle -> Resource Handle)
using GraphRegistry = HandleRegistry<CUgraph, GraphHandle>;
static GraphRegistry graph_registry;
GraphRegistry graph_registry;

// Immutable resource owners for one version of a graph node's parameters.
// Inheriting DeferredCleanupItem lets CUDA's user-object destructor enqueue
Expand Down Expand Up @@ -140,7 +141,7 @@ free_deleter(const void *p) noexcept
GraphBox *
get_box(const GraphHandle &h) noexcept
{
auto *value = reinterpret_cast<const GraphBoxBase *>(h.get());
const auto *value = reinterpret_cast<const GraphBoxBase *>(h.get());
return const_cast<GraphBox *>(static_cast<const GraphBox *>(value));
}

Expand Down Expand Up @@ -254,7 +255,7 @@ struct PreparedAttachmentState {
explicit PreparedAttachmentState(GraphHandle h_graph_) : h_graph(std::move(h_graph_)) {}
};

void
static void
rollback_prepared_attachment(PreparedAttachmentState *state) noexcept
{
if (!state) {
Expand Down Expand Up @@ -290,6 +291,25 @@ struct PreparedChildGraphUpdateState {
}
};

// Deleter of the shared hierarchy: it owns the root CUgraph once the root box
// exists.
static void
release_graph_hierarchy(GraphHierarchy *hierarchy)
{
for (const GraphBox &box : hierarchy->graphs) {
if (box.resource) {
graph_registry.unregister_handle(box.resource);
}
}
GraphBox *root = hierarchy->root();
if (root && root->resource) {
GILReleaseGuard gil;
pw_cuGraphDestroy(root->resource);
}
retry_deferred_cleanup();
delete hierarchy;
}

GraphHandle
create_graph_handle(CUgraph graph)
{
Expand All @@ -298,28 +318,21 @@ create_graph_handle(CUgraph graph)
return {};
}

// clang-format off
auto hierarchy = std::shared_ptr<GraphHierarchy>(
new GraphHierarchy{},
[](GraphHierarchy *hierarchy) {
for (const GraphBox& box : hierarchy->graphs) {
if (box.resource) {
graph_registry.unregister_handle(box.resource);
}
}
GraphBox *root = hierarchy->root();
if (root && root->resource) {
GILReleaseGuard gil;
pw_cuGraphDestroy(root->resource);
}
retry_deferred_cleanup();
delete hierarchy;
}
);
// clang-format on
GraphBox &root = hierarchy->graphs.emplace_back(graph, hierarchy.get());
// Until the root box exists the hierarchy deleter has nothing to destroy,
// so an allocation failure before then must release the graph here.
std::shared_ptr<GraphHierarchy> hierarchy;
GraphBox *root = nullptr;
try {
hierarchy = std::shared_ptr<GraphHierarchy>(new GraphHierarchy{}, release_graph_hierarchy);
root = &hierarchy->graphs.emplace_back(graph, hierarchy.get());
}
catch (...) {
GILReleaseGuard gil;
pw_cuGraphDestroy(graph);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the stream-capture path this graph is not ours to destroy. GraphBuilder.begin_building passes the graph from cuStreamGetCaptureInfo, and the Driver API says of graph_out: "All operations other than destroy and node removal are permitted on the graph while the capture sequence is in progress. This API does not transfer ownership of the graph, which is transferred or destroyed at cuStreamEndCapture."

Before this PR, an allocation failure here leaked the graph, which is what cuda_core/AGENTS.md asks for when ownership cannot be established ("a leak is always preferred to a use-after-free"). The other caller, GraphDefinition, owns its graph. The simplest fix is to drop the destroy and keep the leak; the alternative is to tell create_graph_handle whether the caller owns graph. This is the same point as CodeRabbit's thread on line 332.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think in both cases my agent and CodeRabbit suggested this change because of the noexcept being dropped from GraphBox (as flagged on Windows). It's probably best to revert all of these changes and just suppress the lint and address that separately (if we even need to, which I think you are saying we don't).

throw;
Comment on lines +321 to +332

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Stability & Availability · Quick win

Supported by static analysis

Script executed:

sed -n '460,525p;1050,1130p' cuda_core/cuda/core/graph/_graph_builder.pyx
sed -n '285,340p' cuda_core/cuda/core/_cpp/rt/graph.cpp
git diff 20a4aeeebd2d3226955d3caa142bd19b4d828ced b0a484869e4da84fb317cd2c3e25175144bd1592 -- cuda_core/cuda/core/_cpp/rt/graph.cpp

Repository: NVIDIA/cuda-python

Length of output: 12295


Web query:

site:docs.nvidia.com/cuda/cuda-driver-api cuGraphDestroy active stream capture cuStreamEndCapture graph ownership contract

Result:

<source_evidence>
<source kind="find_in_page">
<title>6.20. Graph Management — CUDA Driver API Reference Manual 13.4 documentation</title>
<location>https://docs.nvidia.com/cuda/cuda-driver-api/cuda_driver_api/group__CUDA__GRAPH.html</location>
<excerpt>citeturn2view1 [wordlim: 200] Crawled: today; Content type: text/html; Source: find({&quot;ref_id&quot;:&quot;turn1view0&quot;,&quot;pattern&quot;:&quot;Destroys a graph&quot;}); Total lines: 3422
L18: CUresult 【4†cuDeviceSetGraphMemAttribute】(CUdevice device, CUgraphMem_attribute attr, void *value)
L19: 
L20: 
L21: Set asynchronous allocation attributes related to graphs.
L22: 
L23: CUresult 【5†cuGraphAddBatchMemOpNode】(CUgraphNode *phGraphNode, CUgraph hGraph, const CUgraphNode *dependencies, size_t numDependencies, const CUDA_BATCH_MEM_OP_NODE_PARAMS *nodeParams)
L24: 
L25: 
L26: Creates a batch memory operation node and adds it to a graph.
L27: CUresult 【6†cuGraphAddChildGraphNode】(CUgraphNode *phGraphNode, CUgraph hGraph, const CUgraphNode *dependencies, size_t numDependencies, CUgraph childGraph)
L28: 
L29: 
L30: Creates a child graph node and adds it to a graph.
L31: 
L32: CUresult 【7†cuGraphAddDependencies】(CUgraph hGraph, const CUgraphNode *from, const CUgraphNode *to, const CUgraphEdgeData *edgeData, size_t numDependencies)
L33: 
L34: 
L35: Adds dependency edges to a graph.
L36: CUresult 【8†cuGraphAddEmptyNode】(CUgraphNode *phGraphNode, CUgraph hGraph, const CUgraphNode *dependencies, size_t numDependencies)
L37: 
L38: 
L39: Creates an empty node and adds it to a graph.
L40: 
L41: CUresult 【9†cuGraphAddEventRecordNode】(CUgraphNode *phGraphNode, CUgraph hGraph, const CUgraphNode *dependencies, size_t numDependencies, CUevent event)
L42: 
L43: 
L44: Creates an event record node and adds it to a graph.
L45: CUresult 【10†cuGraphAddEventWaitNode】(CUgraphNode *phGraphNode, CUgraph hGraph, const CUgraphNode *dependencies, size_t numDependencies, CUevent event)
L46: 
L47: 
L48: Creates an event wait node and adds it to a graph.
L49: 
L50: CUresult 【11†cuGraphAddExternalSemaphoresSignalNode】(CUgraphNode *phGraphNode, CUgraph hGraph, const CUgraphNode *dependencies, size_t numDependencies, const CUDA_EXT_SEM_SIGNAL_NODE_PARAMS *nodeParams)
L51: 
L52: 
L53: Creates an external semaphore signal node and adds it to a graph.
L54: CUresult 【12†cuGraphAddExternalSemaphoresWaitNode】(CUgraphNode *phGraphNode, CUgraph hGraph, const CUgraphNode *dependencies, size_t numDependencies, const CUDA_EXT_SEM_WAIT_NODE_PARAMS *nodeParams)
L55: 
L56: 
L57: Creates an external semaphore wait node and adds it to a graph.
L58: 
L59: CUresult 【13†cuGraphAddHostNode】(CUgraphNode *phGraphNode, CUgraph hGraph, const CUgraphNode *dependencies, size_t numDependencies, const CUDA_HOST_NODE_PARAMS *nodeParams)
L60: 
L61: 
L62: Creates a host execution node and adds it to a graph.
L63: CUresult 【14†cuGraphAddKernelNode】(CUgraphNode *phGraphNode, CUgraph hGraph, const CUgraphNode *dependencies, size_t numDependencies, const CUDA_KERNEL_NODE_PARAMS *nodeParams)
L64: 
L65: 
L66: Creates a kernel execution node and adds it to a graph.
L67: 
L68: CUresult 【15†cuGraphAddMemAllocNode】(CUgraphNode *phGraphNode, CUgraph hGraph, const CUgraphNode *dependencies, size_t numDependencies, CUDA_MEM_ALLOC_NODE_PARAMS *nodeParams)
L69: 
L70: 
L71: Creates an allocation node and adds it to a graph.
L72: CUresult 【16†cuGraphAddMemFreeNode】(CUgraphNode *phGraphNode, CUgraph hGraph, const CUgraphNode *dependencies, size_t numDependencies, CUdeviceptr dptr)
L73: 
L74: 
L75: Creates a memory free node and adds it to a graph.
L76: 
L77: CUresult 【17†cuGraphAddMemcpyNode】(CUgraphNode *phGraphNode, CUgraph hGraph, const CUgraphNode *dependencies, size_t numDependencies, const CUDA_MEMCPY3D *copyParams, CUcontext ctx)
L78: 
L79: 
L80: Creates a memcpy node and adds it to a graph.
L81: CUresult 【18†cuGraphAddMemsetNode】(CUgraphNode *phGraphNode, CUgraph hGraph, const CUgraphNode *dependencies, size_t numDependencies, const CUDA_MEMSET_NODE_PARAMS *memsetParams, CUcontext ctx)
L82: 
L83: 
L84: Creates a memset node and adds it to a graph.
L85: 
L86: CUresult 【19†cuGraphAddNode】(CUgraphNode *phGraphNode, CUgraph hGraph, const CUgraphNode *dependencies, const CUgraphEdgeData *dependencyData, size_t numDependencies, CUgraphNodeParams *nodeParams)
L87: 
L88: 
L89: Adds a node of arbitrary type to a graph.
L90: CUresult 【20†cuGraphAddNode_v3】(CUgraphNode *phGraphNode, CUgraph hGraph, const CUgraphNode *dependencies, const CUgraphEdgeData *dependencyData, size_t numDependencies, CUgraphNodeParams *nodeParams)
L91: 
L92: 
L93: Adds a node of arbitrary type to a graph.
L94: 
L95: CUresult 【21†cuGraphBatchMemOpNodeGetParams】(CUgraphNode hNode, CUDA_BATCH_MEM_OP_NODE_PARAMS *nodeParams_out)
L96: 
L97: 
L98: Returns a batch mem op node&#39;s parameters.
L99: CUresult 【22†cuGraphBatchMemOpNodeSetParams】(CUgraphNode hNode, const CUDA_BATCH_MEM_OP_NODE_PARAMS *nodeParams)
L100: 
L101: 
L102: Sets a batch mem op node&#39;s parameters.
L103: 
L104: CUresult 【23†cuGraphChildGraphNodeGetGraph】(CUgraphNode hNode, CUgraph *phGraph)
L105: 
L106: 
L107: Gets a handle to the embedded graph of a child graph node.
L108: 
L109: CUresult 【24†cuGraphClone】(CUgraph *phGraphClone, CUgraph originalGraph)
L110: 
L111: 
L112: Clones a graph.
L113: CUresult 【25†cuGraphConditionalHandleCreate】(CUgraphConditionalHandle *pHandle_out, CUgraph hGraph, CUcontext ctx, unsigned int defaultLaunchValue, unsigned int flags)
L114: 
L115: 
L116: Create a conditional handle.
L117: 
L118: CUresult 【26†cuGraphCreate】(CUgraph *phGraph, unsigned int flags)
L119: 
L120: 
L121: Creates a graph.
L122: 
L123: CUresult 【27†cuGraphDebugDotPrint】(CUgraph hGraph, const char *path, unsigned int flags)
L124: 
L125: 
L126: Write a DOT file describing graph structure.
L127: 
L128: CUresult 【28†cuGraphDestroy】(CUgraph hGraph)
L129: 
L130: 
L131: Destroys a graph.
L132: CUresult 【29†cuGraphDestroyNode】(CUgraphNode hNode)
L133: 
L134: 
L135: Remove a node from the graph.
L136: 
L137: CUresult 【30†cuGraphEventRecordNodeGetEvent】(CUgraphNode hNode, CUevent *event_out)
L138: 
L139: 
L140: Returns the event associated with an event record node.
L141: 
L142: CUresult 【31†cuGraphEventRecordNodeSetEvent】(CUgraphNode hNode, CUevent event)
L143: 
L144: 
L145: Sets an event record node&#39;s event.
L146: 
L147: CUresult 【32†cuGraphEventWaitNodeGetEvent】(CUgraphNode hNode, CUevent *event_out)
L148: 
L149: 
L150: Returns the event associated with an event wait node.
L151: CUresult 【33†cuGraphEventWaitNodeSetEvent】(CUgraphNode hNode, CUevent event)
L152: 
L153: 
L154: Sets an event wait node&#39;s event.
L155: 
L156: CUresult 【34†cuGraphExecBatchMemOpNodeSetParams】(CUgraphExec hGraphExec, CUgraphNode hNode, const CUDA_BATCH_MEM_OP_NODE_PARAMS *nodeParams)
L157: 
L158: 
L159: Sets the parameters for a batch mem op node in the given graphExec.
L160: 
L161: CUresult 【35†cuGraphExecChildGraphNodeSetParams】(CUgraphExec hGraphExec, CUgraphNode hNode, CUgraph childGraph)
L207: CUresult 【46†cuGraphExecMemsetNodeSetParams】(CUgraphExec hGraphExec, CUgraphNode hNode, const CUDA_MEMSET_NODE_PARAMS *memsetParams, CUcontext ctx)
L208: 
L209: 
L210: Sets the parameters for a memset node in the given graphExec.
L211: 
L212: CUresult 【47†cuGraphExecNodeSetParams】(CUgraphExec hGraphExec, CUgraphNode hNode, CUgraphNodeParams *nodeParams)
L213: 
L214: 
L215: Update a graph node&#39;s parameters in an instantiated graph.
L216: CUresult 【48†cuGraphExecUpdate】(CUgraphExec hGraphExec, CUgraph hGraph, CUgraphExecUpdateResultInfo *resultInfo)
L217: 
L218: 
L219: Check whether an executable graph can be updated with a graph and perform the update if possible.
L220: 
L221: CUresult 【49†cuGraphExternalSemaphoresSignalNodeGetParams】(CUgraphNode hNode, CUDA_EXT_SEM_SIGNAL_NODE_PARAMS *params_out)
L222: 
L223: 
L224: Returns an external semaphore signal node&#39;s parameters.
L236: Sets an external semaphore wait node&#39;s parameters.
L237: 
L238: CUresult 【53†cuGraphGetEdges】(CUgraph hGraph, CUgraphNode *from, CUgraphNode *to, CUgraphEdgeData *edgeData, size_t *numEdges)
L239: 
L240: 
L241: Returns a graph&#39;s dependency edges.
L242: 
L243: CUresult 【54†cuGraphGetId】(CUgraph hGraph, unsigned int *graphId)
L244: 
L245: 
L246: Returns the id of a given graph.
L247: 
L248: CUresult 【55†cuGraphGetNodes】(CUgraph hGraph, CUgraphNode *nodes, size_t *numNodes)
L249: 
L250: 
L251: Returns a graph&#39;s nodes.
L252: CUresult 【56†cuGraphGetRootNodes】(CUgraph hGraph, CUgraphNode *rootNodes, size_t *numRootNodes)
L253: 
L254: 
L255: Returns a graph&#39;s root nodes.
L256: 
L257: CUresult 【57†cuGraphHostNodeGetParams】(CUgraphNode hNode, CUDA_HOST_NODE_PARAMS *nodeParams)
L258: 
L259: 
L260: Returns a host node&#39;s parameters.
L261: 
L262: CUresult 【58†cuGraphHostNodeSetParams】(CUgraphNode hNode, const CUDA_HOST_NODE_PARAMS *nodeParams)
L263: 
L264: 
L265: Sets a host node&#39;s parameters.
L266: 
L267: CUresult 【59†cuGraphInstantiate】(CUgraphExec *phGraphExec, CUgraph hGraph, unsigned long long flags)
L268: Creates an executable graph from a graph.
L269: 
L270: CUresult 【60†cuGraphInstantiateWithParams】(CUgraphExec *phGraphExec, CUgraph hGraph, CUDA_GRAPH_INSTANTIATE_PARAMS *instantiateParams)
L271: 
L272: 
L273: Creates an executable graph from a graph.
L274: 
L275: CUresult 【61†cuGraphKernelNodeCopyAttributes】(CUgraphNode dst, CUgraphNode src)
L276: 
L277: 
L278: Copies attributes from source node to destination node.
L279: 
L280: CUresult 【62†cuGraphKernelNodeGetAttribute】(CUgraphNode hNode, CUkernelNodeAttrID attr, CUkernelNodeAttrValue *value_out)
L357: CUresult 【79†cuGraphNodeGetParams】(CUgraphNode hNode, CUgraphNodeParams *nodeParams)
L358: 
L359: 
L360: Return a graph node&#39;s parameters.
L361: 
L362: CUresult 【80†cuGraphNodeGetToolsId】(CUgraphNode hNode, unsigned long long *toolsNodeId)
L363: 
L364: 
L365: Returns an id used by tools to identify a given node.
L366: 
L367: CUresult 【81†cuGraphNodeGetType】(CUgraphNode hNode, CUgraphNodeType *type)
L368: 
L369: 
L370: Returns a node&#39;s type.
L371: 
L372: CUresult 【82†cuGraphNodeSetEnabled】(CUgraphExec hGraphExec, CUgraphNode hNode, unsigned int isEnabled)
L373: Enables or disables the specified node in the given graphExec.
L374: 
L375: CUresult 【83†cuGraphNodeSetParams】(CUgraphNode hNode, CUgraphNodeParams *nodeParams)
L376: 
L377: 
L378: Update a graph node&#39;s parameters.
L379: 
L380: CUresult 【84†cuGraphNodeSetParams_v2】(CUgraphNode hNode, CUgraphNodeParams *nodeParams)
L381: 
L382: 
L383: Update a graph node&#39;s parameters.
L384: 
L385: CUresult 【85†cuGraphReleaseUserObject】(CUgraph graph, CUuserObject object, unsigned int count)
L386: 
L387: 
L388: Release a user object reference from a graph.
L389: CUresult 【86†cuGraphRemoveDependencies】(CUgraph hGraph, const CUgraphNode *from, const CUgraphNode *to, const CUgraphEdgeData *edgeData, size_t numDependencies)
L390: 
L391: 
L392: Removes dependency edges from a graph.
L393: 
L394: CUresult 【87†cuGraphRetainUserObject】(CUgraph graph, CUuserObject object, unsigned int count, unsigned int flags)
L395: 
L396: 
L397: Retain a reference to a user object from a graph.
L398: 
L399: CUresult 【88†cuGraphUpload】(CUgraphExec hGraphExec, CUstream hStream)
L400: 
L401: 
L402: Uploads an executable graph in a stream.
L1341:   * flags – - Flags from CUgraphDebugDot_flags for specifying which additional node information to write
L1342: 
L1343: Returns
L1344: 
L1345: 
L1346: 【101†CUDA_SUCCESS】, 【112†CUDA_ERROR_INVALID_VALUE】, 【169†CUDA_ERROR_OPERATING_SYSTEM】
L1347: 
L1348: 【93†CUresult】 cuGraphDestroy(【106†CUgraph】 hGraph)【170†】
L1349: 
L1350: 
L1351: Destroys a graph.
L1352: 
L1353: Destroys the graph specified by `hGraph`, as well as all of its nodes.
L1354: 
L1355: See also
L1356: 
L1357: 【26†cuGraphCreate】
L1358: 
L1359: Note
L1360: 
L1361: Graph objects are not threadsafe. More here.
L1362: 
L1363: Note
</excerpt>
</source>
<source kind="find_in_page">
<title>6.36. Stream Management — CUDA Driver API Reference Manual 13.4 documentation</title>
<location>https://docs.nvidia.com/cuda/cuda-driver-api/cuda_driver_api/group__CUDA__STREAM.html</location>
<excerpt>citeturn2view2 [wordlim: 200] Crawled: today; Content type: text/html; Source: find({&quot;ref_id&quot;:&quot;turn1view1&quot;,&quot;pattern&quot;:&quot;cuStreamEndCapture(&quot;}); Total lines: 1059
L59: CUresult 【13†cuStreamEndCapture】(CUstream hStream, CUgraph *phGraph)
L60: 
L61: 
L62: Ends capture on a stream, returning the captured graph.
L63: 
L64: CUresult 【14†cuStreamEndCaptureToCig】(CUstream hStream)
L65: 
L66: 
L67: Ends CIG capture on a stream.
L68: 
L69: CUresult 【15†cuStreamGetAttribute】(CUstream hStream, CUstreamAttrID attr, CUstreamAttrValue *value_out)
L70: 
L71: 
L72: Queries stream attribute.
L263: If `mode` is not CU_STREAM_CAPTURE_MODE_RELAXED, 【13†cuStreamEndCapture】 must be called on this stream from the same thread.
L264: 
L265: See also
L266: 
L267: 【10†cuStreamCreate】, 【23†cuStreamIsCapturing】, 【13†cuStreamEndCapture】, 【29†cuThreadExchangeStreamCaptureMode】
L268: 
L269: Note
L270: 
L271: Kernels captured using this API must not use texture and surface references. Reading or writing through any texture or surface reference is undefined behavior. This restriction does not apply to texture and surface objects.
L272: 
L273: Note
L328: Begin graph capture on `hStream`, placing new nodes into an existing graph. When a stream is in capture mode, all operations pushed into the stream will not be executed, but will instead be captured into `hGraph`. The graph will not be instantiable until the user calls 【13†cuStreamEndCapture】.
L329: Capture may not be initiated if `stream` is CU_STREAM_LEGACY. Capture must be ended on the same stream in which it was initiated, and it may only be initiated if the stream is not already in capture mode. The capture mode may be queried via 【23†cuStreamIsCapturing】. A unique id representing the capture sequence may be queried via cuStreamGetCaptureInfo.
L330: 
L331: If `mode` is not CU_STREAM_CAPTURE_MODE_RELAXED, 【13†cuStreamEndCapture】 must be called on this stream from the same thread.
L332: 
L333: See also
L334: cuStreamBeginCapture, 【10†cuStreamCreate】, 【23†cuStreamIsCapturing】, 【13†cuStreamEndCapture】, 【29†cuThreadExchangeStreamCaptureMode】, cuGraphAddNode
L335: 
L336: Note
L337: 
L338: Kernels captured using this API must not use texture and surface references. Reading or writing through any texture or surface reference is undefined behavior. This restriction does not apply to texture and surface objects.
L339: 
L340: Note
L341: 
L342: Note that this function may also return error codes from previous, asynchronous launches.
L343: 
L344: Parameters
L377: 【10†cuStreamCreate】, cuStreamBeginCapture, 【23†cuStreamIsCapturing】, 【13†cuStreamEndCapture】, 【29†cuThreadExchangeStreamCaptureMode】
L378: 
L379: Note
L380: 
L381: Any user objects associated with `graph` will be released prior to the recapture.
L382: 
L383: Note
L384: 
L385: Note that this function may also return error codes from previous, asynchronous launches.
L386: 
L387: Parameters
L388: 
L389: 
L390:   * hStream – - Stream in which to initiate capture
L627:   * graph_out – - Optional location to return the graph being captured into. All operations other than destroy and node removal are permitted on the graph while the capture sequence is in progress. This API does not transfer ownership of the graph, which is transferred or destroyed at 【13†cuStreamEndCapture】. Note that the graph handle may be invalidated before end of capture for certain errors.
L628: Nodes that are or become unreachable from the original stream at 【13†cuStreamEndCapture】 due to direct actions on the graph do not trigger 【99†CUDA_ERROR_STREAM_CAPTURE_UNJOINED】.
L629:   * dependencies_out – - Optional location to store a pointer to an array of nodes. The next node to be captured in the stream will depend on this set of nodes, absent operations such as event wait which modify this set. The array pointer is valid until the next API call which operates on the stream or until the capture is terminated. The node handles may be copied out and are valid until they or the graph is destroyed.
L810:   * 【112†CU_STREAM_CAPTURE_STATUS_NONE】: The stream is not capturing.
L811: 
L812:   * 【97†CU_STREAM_CAPTURE_STATUS_ACTIVE】: The stream is capturing.
L813: 
L814:   * 【113†CU_STREAM_CAPTURE_STATUS_INVALIDATED】: The stream was capturing but an error has invalidated the capture sequence. The capture sequence must be terminated with 【13†cuStreamEndCapture】 on the stream where it was initiated in order to continue using `hStream`.
L997: Therefore, stream capture places restrictions on API calls that can be made within or concurrently to a cuStreamBeginCapture-【13†cuStreamEndCapture】 sequence. This behavior can be controlled via this API and flags to cuStreamBeginCapture.
L998: 
L999: A thread’s mode is one of the following:
</excerpt>
</source>
<source kind="open_page">
<title>6.20. Graph Management — CUDA Driver API Reference Manual 13.4 documentation</title>
<location>https://docs.nvidia.com/cuda/cuda-driver-api/cuda_driver_api/group__CUDA__GRAPH.html</location>
<excerpt>citeturn1view0 [wordlim: 200] Crawled: today; Content type: text/html; Source: open({&quot;ref_id&quot;:&quot;turn0search1&quot;,&quot;lineno&quot;:null}); Total lines: 3422
L2385: Instantiates `hGraph` as an executable graph. The graph is validated for any structural constraints or intra-node constraints which were not previously validated. If instantiation is successful, a handle to the instantiated graph is returned in `phGraphExec`.
L2386: 
L2387: The `flags` parameter controls the behavior of instantiation and subsequent graph launches. Valid flags are:
L2388:   * 【143†CUDA_GRAPH_INSTANTIATE_FLAG_AUTO_FREE_ON_LAUNCH】, which configures a graph containing memory allocation nodes to automatically free any unfreed memory allocations before the graph is relaunched.
L2389:   * 【208†CUDA_GRAPH_INSTANTIATE_FLAG_DEVICE_LAUNCH】, which configures the graph for launch from the device. If this flag is passed, the executable graph handle returned can be used to launch the graph from both the host and device. This flag can only be used on platforms which support unified addressing. This flag cannot be used in conjunction with 【143†CUDA_GRAPH_INSTANTIATE_FLAG_AUTO_FREE_ON_LAUNCH】.
L2390:   * 【209†CUDA_GRAPH_INSTANTIATE_FLAG_USE_NODE_PRIORITY】, which causes the graph to use the priorities from the per-node attributes rather than the priority of the launch stream during execution. Note that priorities are only available on kernel nodes, and are copied from stream priority during stream capture.
L2391: If `hGraph` contains any allocation or free nodes, there can be at most one executable graph in existence for that graph at a time. An attempt to instantiate a second executable graph before destroying the first with 【36†cuGraphExecDestroy】 will result in an error. The same also applies if `hGraph` contains any device-updatable kernel nodes.
L2392: 
L2393: If `hGraph` contains kernels which call device-side cudaGraphLaunch() from multiple contexts, this will result in an error.
L2394: Graphs instantiated for launch on the device have additional restrictions which do not apply to host graphs:
L2395: 
L2396:   * The graph’s nodes must reside on a single context.
L2397: 
L2398:   * The graph can only contain kernel nodes, memcpy nodes, memset nodes, and child graph nodes.
L2399: 
L2400:   * The graph cannot be empty and must contain at least one kernel, memcpy, or memset node. Operation-specific restrictions are outlined below.
L2401: 
L2402:   * Kernel nodes:
L2403: 
L2404:     * Use of CUDA Dynamic Parallelism is not permitted.
L2405:     * Cooperative launches are permitted as long as MPS is not in use.
L2406: 
L2407:   * Memcpy nodes:
L2408: 
L2409:     * Only copies involving device memory and/or pinned device-mapped host memory are permitted.
L2410: 
L2411:     * Copies involving CUDA arrays are not permitted.
L2412: 
L2413:     * Both operands must be accessible from the current context, and the current context must match the context of other nodes in the graph.
L2414: 
L2415: See also
L2416: 
L2417: cuGraphInstantiate, 【26†cuGraphCreate】, 【88†cuGraphUpload】, 【66†cuGraphLaunch】, 【36†cuGraphExecDestroy】
L2418: 
L2419: Note
L2420: Graph objects are not threadsafe. More here.
L2421: 
L2422: Note
L2423: 
L2424: Note that this function may also return error codes from previous, asynchronous launches.
L2425: 
L2426: Parameters
L2427: 
L2428: 
L2429:   * phGraphExec – - Returns instantiated graph
L2430: 
L2431:   * hGraph – - Graph to instantiate
L2432: 
L2433:   * flags – - Flags to control instantiation. See 【210†CUgraphInstantiate_flags】.
L2434: 
L2435: Returns
L2436: 
L2437: 
L2438: 【101†CUDA_SUCCESS】, 【109†CUDA_ERROR_DEINITIALIZED】, 【110†CUDA_ERROR_NOT_INITIALIZED】, 【112†CUDA_ERROR_INVALID_VALUE】
L2439: 【93†CUresult】 cuGraphInstantiateWithParams(【176†CUgraphExec】 *phGraphExec, 【106†CUgraph】 hGraph, 【211†CUDA_GRAPH_INSTANTIATE_PARAMS】 *instantiateParams)【212†】
L2440: 
L2441: 
L2442: Creates an executable graph from a graph.
L2443: Instantiates `hGraph` as an executable graph according to the `instantiateParams` structure. The graph is validated for any structu…[truncated]</excerpt>
</source>
<source kind="open_page">
<title>6.36. Stream Management — CUDA Driver API Reference Manual 13.4 documentation</title>
<location>https://docs.nvidia.com/cuda/cuda-driver-api/cuda_driver_api/group__CUDA__STREAM.html</location>
<excerpt>citeturn1view1 [wordlim: 200] Crawled: today; Content type: text/html; Source: open({&quot;ref_id&quot;:&quot;turn0search0&quot;,&quot;lineno&quot;:null}); Total lines: 1059
L619: Parameters
L620: 
L621: 
L622:   * hStream – - The stream to query
L623: 
L624:   * captureStatus_out – - Location to return the capture status of the stream; required
L625: 
L626:   * id_out – - Optional location to return an id for the capture sequence, which is unique over the lifetime of the process
L627:   * graph_out – - Optional location to return the graph being captured into. All operations other than destroy and node removal are permitted on the graph while the capture sequence is in progress. This API does not transfer ownership of the graph, which is transferred or destroyed at 【13†cuStreamEndCapture】. Note that the graph handle may be invalidated before end of capture for certain errors.
L628: Nodes that are or become unreachable from the original stream at 【13†cuStreamEndCapture】 due to direct actions on the graph do not trigger 【99†CUDA_ERROR_STREAM_CAPTURE_UNJOINED】.
L629:   * dependencies_out – - Optional location to store a pointer to an array of nodes. The next node to be captured in the stream will depend on this set of nodes, absent operations such as event wait which modify this set. The array pointer is valid until the next API call which operates on the stream or until the capture is terminated. The node handles may be copied out and are valid until they or the graph is destroyed.
L630: The driver-owned array may also be passed directly to APIs that operate on the graph (not the stream) without copying.
L631:   * edgeData_out – - Optional location to store a pointer to an array of graph edge data. This array parallels `dependencies_out`; the next node to be added has an edge to `dependencies_out`[i] with annotation `edgeData_out`[i] for each `i`. The array pointer is valid until the next API call which operates on the stream or until the capture is terminated.
L632: 
L633:   * numDependencies_out – - Optional location to store the size of the array returned in dependencies_out.
L634: 
L635: Returns
L636: 【41†CUDA_SUCCESS】, 【61†CUDA_ERROR_INVALID_VALUE】, 【96†CUDA_ERROR_STREAM_CAPTURE_IMPLICIT】, 【98†CUDA_ERROR_LOSSY_QUERY】
L637: 
L638: 【37†CUresult】 cuStreamGetCtx(【38†CUstream】 hStream, 【100†CUcontext】 *pctx)【101†】
L639: 
L640: 
L641: Query the context associated with a stream.
L642: 
L643: Returns the CUDA context that the stream is associated with.
L644: If the stream was created via the API 【79†cuGreenCtxStreamCreate】, the returned context is equivalent to the one returned by 【102†cuCtxFromGreenCtx()】 on the green context associated with the stream at creation time.
L645: 
L646: The stream handle `hStream` can refer to any of the following:
L647:   * a stream created via any of the CUDA driver APIs such as 【10†cuStreamCreate】 and 【11†cuStreamCreateWithPriority】, or their runtime API equivalents such as ::cudaStreamCreate, ::cudaStreamCreateWithFlags and ::cudaStreamCreateWithPriority. The returned context is the context that was active in the calling thread when the stream was created. Passing an invalid handle will result in undefined behavior.
L648:   * any of the special streams such as the NULL stream, 【95†CU_STREAM_LEGACY】 and 【103†CU_STREAM_PER_THREAD】. The runtime API equivalents of these are also accepted, which are NULL, ::cudaStreamLegacy and ::cudaStreamPerThread respectively. Specifying any of the special handles will return the context current to the calling thread. If no context is current to the calling thread, 【48†CUDA_ERROR_INVALID_CONTEXT】 is returned.
L649: 
L650: See also
L651: cuStreamDestroy, 【11†cuStreamCreateWithPriority】, 【22†cuStreamGetPriority】, 【20†cuStreamGetFlags】, 【19†cuStreamGetDevice】 【28†cuStreamWaitEvent】, 【24†cuStreamQuery】, 【26†cuStreamSynchronize】, 【3†cuStreamAddCallback】, ::cudaStreamCreate, ::cudaStreamCreateWithFlags
L652: 
L653: Note
L654: 
L655: Note that this functio…[truncated]</excerpt>
</source>
<source kind="search">
<title>6.36. Stream Management — CUDA Driver API Reference Manual 13.4 documentation</title>
<location>https://docs.nvidia.com/cuda/cuda-driver-api/cuda_driver_api/group__CUDA__STREAM.html</location>
<excerpt>citeturn0search0 [wordlim: 200] Crawled: 2 weeks ago; cuStreamCreate, cuStreamBeginCapture, cuStreamIsCapturing, cuGraphDestroy ...   * a stream created via any of the CUDA driver APIs such as cuStreamCreate, cuStreamCreateWithPriority and cuGreenCtxStreamCreate, or their runtime API equivalents such as ::cudaStreamCreate, ::cudaStreamCreateWithFlags and ::cudaStreamCreateWithPriority. ... Nodes that are removed from the dependency set via this API do not result in CUDA_ERROR_STREAM_CAPTURE_UNJOINED if they are unreachable from the stream at cuStreamEndCapture.
    

Begins graph capture on a stream to an existing graph.

CUresult cuStreamBeginRecaptureToGraph(CUstream hStream, CUstreamCaptureMode mode, CUgraph hGraph, CUgraphRecaptureCallback callbackFunc, void *userData)
    

Begin graph capture on a stream to an existing graph.

CUresult cuStreamCopyAttributes(CUstream dst, CUstream src)
    

Copies attributes from source stream to destination stream.

CUresult cuStreamCreate(CUstream *phStream, unsigned int Flags)
    

Create a stream.

CUresult cuStreamCreateWithPriority(CUstream *phStream, unsigned int flags, int priority)
    

Create a stream with the given priority.

CUresult cuStreamDestroy(CUstream hStream)
    

Destroys a stream.

CUresult cuStreamEndCapture(CUstream hStream, CUgraph *phGraph)
    

Ends capture on a stream, returning the captured graph.

CUresult cuStreamEndCaptureToCig(CUstream hStream)
    

Ends CIG capture on a stream.

CUresult cuStreamGetAttribute(CUstream hStream, CUstreamAttrID attr, CUstreamAttrValue *value_out)

CUresult cuStreamBeginCapture(CUstream hStream, CUstreamCaptureMode mode)
    

Begins graph capture on a stream.

Begin graph capture on `hStream`. When a stream is in capture mode, all operations pushed into the stream will not be executed, but will instead be captured into a graph, which will be returned via cuStreamEndCapture. Capture may not be initiated if `stream` is CU_STREAM_LEGACY. Capture must be ended on the same stream in which it was initiated, and it may only be initiated if the stream is not already in capture mode. The capture mode may be queried via cuStreamIsCapturing. A unique id representing the capture sequence may be queried via cuStreamGetCaptureInfo.

If `mode` is not CU_STREAM_CAPTURE_MODE_RELAXED, cuStreamEndCapture must be called on this stream from the same thread.

See also

cuStreamCreate, cuStreamIsCapturing, cuStreamEndCapture, cuThreadExchangeStreamCaptureMode
Parameters
    

  * hStream – - Stream in which to initiate capture to CIG

  * streamCigCaptureParams – - CIG capture parameters

Returns
    

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_NOT_SUPPORTED, CUDA_ERROR_INVALID_CONTEXT, CUDA_ERROR_INVALID_VALUE,

CUresult cuStreamBeginCaptureToGraph(CUstream hStream, CUgraph hGraph, const CUgraphNode *dependencies, const CUgraphEdgeData *dependencyData, size_t numDependencies, CUstreamCaptureMode mode)
    

Begins graph capture on a stream to an existing graph.

Begin graph capture on `hStream`, placing new nodes into an existing graph. When a stream is in capture mode, all operations pushed into the stream will not be executed, but will instead be captured into `hGraph`. The graph will not be instantiable until the user calls cuStreamEndCapture.

Capture may not be initiated if `stream` is CU_STREAM_LEGACY. 
CUresult cuStreamDestroy(CUstream hStream)
    

Destroys a stream.

Destroys the stream specified by `hStream`.

In case the device is still doing work in the stream `hStream` when cuStreamDestroy() is called, the function will return immediately and the resources associated with `hStream` will be released automatically once the device has completed all work in `hStream`.

See also

cuStreamCreate, cuStreamWaitEvent, cuStreamQuery, cuStreamSynchronize, cuStreamAddCallback, ::cudaStreamDestroy

Note

Note that this function may also return error codes from previous, asynchronous lau…[truncated]</excerpt>
</source>
<omitted_sources count="15">Additional source records were omitted from this bounded result.</omitted_sources>
</source_evidence>

Citations:

- 1: https://docs.nvidia.com/cuda/cuda-driver-api/cuda_driver_api/group__CUDA__STREAM.html
- 2: https://docs.nvidia.com/cuda/cuda-driver-api/cuda_driver_api/group__CUDA__STREAM.html

Script executed:

set -o pipefail
printf '%s\n' '--- repository references to capture graph ownership and end capture ---'
rg -n -F --glob '*.{pyx,pxd,cpp,hpp,rst,md,py}' -e 'cuStreamEndCapture' -e 'cuGraphDestroy' -e 'invalidate_root_graph_state' -e 'end_building' -e 'capture graph' cuda_core | head -240
printf '%s\n' '--- relevant base/head source blocks ---'
for rev in 20a4aeeebd2d3226955d3caa142bd19b4d828ced b0a484869e4da84fb317cd2c3e25175144bd1592; do
  printf '\\nREV %s\\n' "$rev"
  git show "$rev:cuda_core/cuda/core/graph/_graph_builder.pyx" | nl -ba | sed -n '1060,1125p'
  git show "$rev:cuda_core/cuda/core/_cpp/rt/graph.cpp" | nl -ba | sed -n '285,345p'
done

Repository: NVIDIA/cuda-python

Length of output: 27519


Script executed:

set -o pipefail
printf '%s\n' '--- create_graph_handle callers ---'
rg -n -F --glob '*.{pyx,pxd,cpp,hpp}' -e 'create_graph_handle(' cuda_core/cuda
printf '%s\n' '--- primary begin/close/deallocation paths ---'
sed -n '440,575p' cuda_core/cuda/core/graph/_graph_builder.pyx
sed -n '1025,1128p' cuda_core/cuda/core/graph/_graph_builder.pyx
printf '%s\n' '--- GraphDefinition caller ---'
sed -n '90,135p' cuda_core/cuda/core/graph/_graph_definition.pyx
printf '%s\n' '--- driver wrapper declaration and implementation references ---'
rg -n -F --glob '*.{hpp,cpp}' -e 'WarnOnFailure<p_cuGraphDestroy>' -e 'struct WarnOnFailure' -e 'class WarnOnFailure' -e 'pw_cuGraphDestroy' cuda_core/cuda/core/_cpp/rt

Repository: NVIDIA/cuda-python

Length of output: 12966


important: End the active capture before releasing its graph.

create_graph_handle can call cuGraphDestroy while the graph still belongs to an active primary capture. CUDA transfers or destroys that graph at cuStreamEndCapture and does not permit graph destruction during capture.

If the driver rejects the destroy call, begin_building() leaves _h_graph unset. The later GB_end_capture() can return a non-null graph, but its non-null branch only marks the capture ended and does not destroy the returned graph. The graph can leak.

Handle capture cleanup in the primary-capture caller. End the capture before destroying the returned graph. Keep the direct destruction path for GraphDefinition, which passes a non-capture graph.

}

GraphHandle h_graph(hierarchy, &root.resource);
GraphHandle h_graph(hierarchy, &root->resource);
graph_registry.register_handle(graph, h_graph);
return h_graph;
}
Expand Down Expand Up @@ -389,7 +402,7 @@ graph_prepare_child_graph_update(
return CUDA_SUCCESS;
}

void
static void
publish_child_graph_update(PreparedChildGraphUpdateState &state, GraphHandle *out_child)
{
GraphBox *parent = get_box(state.h_parent);
Expand Down
4 changes: 2 additions & 2 deletions cuda_core/cuda/core/_cpp/rt/graph_exec.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ struct ExecAttachmentStaging {
if (!h_source || !accumulator) {
return CUDA_SUCCESS;
}
const CUuserObject object = accumulator->object;
CUuserObject object = accumulator->object;
const GraphHandle source = std::move(h_source);
accumulator = nullptr;
GILReleaseGuard gil;
Expand Down Expand Up @@ -154,7 +154,7 @@ struct PreparedExecAttachmentState {
}
};

void
static void
rollback_prepared_exec_attachment(PreparedExecAttachmentState *state) noexcept
{
if (!state) {
Expand Down
Loading
Loading