Repository navigation
perf(cuda): release closed mappings off the decode thread - #10718
Conversation
Merging this PR will improve performance by 25.2%3 warnings, 1 note
Performance Changes
Tip Curious why performance improved? Comment Comparing Footnotes
|
8ba0f78 to
d91362d
Compare
8ed5762 to
18c5897
Compare
robert3005
left a comment
There was a problem hiding this comment.
I wonder if there's a thing that already runs on a per thread basis that we can move this to instead of having a thread
18c5897 to
a149621
Compare
d91362d to
8845ef7
Compare
a149621 to
34a45d9
Compare
Move closed readers' copy-stream waits and mapping release onto a background worker. Drain queued releases before explicit cache eviction and retain mappings when synchronization fails. Include the release-before-return regression test. Original-commit: e1b178661a Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
8845ef7 to
5fadeca
Compare
Since #10718, a closed reader's mapping is released on a background thread after synchronizing the reader's copy stream. That stream comes from the session's pool and is shared with other scans, so the release also waited for any work they queued after the reader closed. Releases run one at a time, so a long kernel on one pooled stream held back every later release and `release_file_mappings`. Record an event when the reader closes and wait for that instead, which covers exactly the reader's copies. If the event cannot be created, the release still synchronizes the stream. Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
Since #10718, a closed reader's mapping is released on a background thread after synchronizing the reader's copy stream. That stream comes from the session's pool and is shared with other scans, so the release also waited for any work they queued after the reader closed. Releases run one at a time, so a long kernel on one pooled stream held back every later release and `release_file_mappings`. Record an event when the reader closes and wait for that instead, which covers exactly the reader's copies. If the event cannot be created, the release still synchronizes the stream. Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
Since #10718, a closed reader's mapping is released on a background thread after synchronizing the reader's copy stream. That stream comes from the session's pool and is shared with other scans, so the release also waited for any work they queued after the reader closed. Releases run one at a time, so a long kernel on one pooled stream held back every later release and `release_file_mappings`. Record an event when the reader closes and wait for that instead, which covers exactly the reader's copies. If the event cannot be created, the release still synchronizes the stream. Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
Problem
Closing a mapped reader can stall decoding while waiting for GPU copies and releasing page tables.
Solution
Move synchronization and mapping release to a background worker; drain queued releases before explicit cache eviction.
Assessment
Falls back to synchronous release if queuing fails and retains mappings if completion is unknown. Depends on #10717.