Skip to content

Use an increased release threshold for the default async pool when using DeviceMemoryResource #771

New issue
New issue
Closed
#875
Closed
Use an increased release threshold for the default async pool when using DeviceMemoryResource#771
#875
Assignees
leofangcopilot-swe-agent
Labels
P0High priority - Must do!cuda.coreEverything related to the cuda.core moduleenhancementAny code-related improvements
Milestone
cuda.core beta 7

Description

@shwina
shwina
opened on Jul 21, 2025

As reported in #756 (comment), allocation via DeviceMemoryResource can be very slow when there are no active allocations. This is due to the default pool being configured with a release threshold of 0.

A fix for this could be for DeviceMemoryResource to configure the default pool to have a larger (or maximum) release threshold via cuMemPoolSetAttribute.

Activity

  1. github-project-automation
    added this to CCCLon Jul 21, 2025
  2. github-project-automation
    moved this to Todo in CCCLon Jul 21, 2025
  3. github-actions
    added
    triageNeeds the team's attention
    on Jul 21, 2025
  4. leofang
    self-assigned this
    on Jul 21, 2025
  5. leofang
    added
    enhancementAny code-related improvements
    P0High priority - Must do!
    cuda.coreEverything related to the cuda.core module
    and removed
    triageNeeds the team's attention
    on Jul 21, 2025
  6. leofang
    added this to the cuda.core beta 6 milestone on Jul 21, 2025
  7. leofang
    modified the milestones: cuda.core beta 6, cuda.core beta 7 on Aug 7, 2025

  8. leofang commented on Aug 21, 2025

    @leofang
    leofang
    on Aug 21, 2025
    Member

    We need to upstream this workaround
    https://github.com/NVIDIA/cccl/blob/cf69aecbbadcd80a1107b0db41896b9d697301c9/python/cuda_cccl/cuda/cccl/parallel/experimental/_utils/temp_storage_buffer.py#L21-L33
    done in cuda-cccl back to cuda-core. This should be done in DeviceMemoryResource as noted. This performance improvement should be release-noted.

  9. leofang
    assigned
    Copilot
    on Aug 21, 2025
  10. leofang
    closed this as completedin #875on Aug 21, 2025
  11. github-project-automation
    moved this from Todo to Done in CCCLon Aug 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

P0High priority - Must do!cuda.coreEverything related to the cuda.core moduleenhancementAny code-related improvements

Type

No type

Projects

  • CCCL
    Status
    Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions