Repository navigation
perf(cuda): copy cached file pages directly on the GPU - #10716
Conversation
Merging this PR will regress 1 benchmark3 warnings, 1 note
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
d006295 to
715005f
Compare
715005f to
cbbedfc
Compare
On devices that read pageable memory through host page tables, copy resident mapped-file ranges directly into device memory. Preserve staged reads for uncached ranges and direct I/O, source and destination alignment, cancellation safety, and mapping lifetime guards. Use the existing file-read chunk size so this feature is independent of chunk-size tuning. Original-commit: b546d93814 Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
cbbedfc to
5d8b5f3
Compare
## Problem Repeated scans rebuild file mappings and page tables; caching them also needs explicit support for cold-cache eviction. ## Solution Cache mappings by file identity/version and add `release_file_mappings()` and `vx_cuda_release_file_mappings()`. ## Assessment Callers can release cached mappings before eviction while active readers retain ownership. Depends on #10716. Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
Since #10716, opening a file for CUDA reads on devices that read pageable memory maps the file so the device can copy cached pages directly. A failed `mmap` failed the whole open, although the mapping only speeds up reads that staging serves as well. Files on filesystems without shared mappings, such as some FUSE mounts, could therefore no longer be read at all on those devices. Fall back to staged reads when the file cannot be mapped. Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
`CudaOpenOptions::open_path` opened the path three times: for the footer and zone maps, for the pooled data reader, and, since #10716, for the mapping the device copies cached pages from. A file atomically replaced at that path between the opens, as writers commonly publish files, could therefore pair one file's footer with another's segments, or serve resident ranges from one file's mapping and other ranges from the other file. Files with compatible layouts then decoded wrong data without any error. Open the file once and give that handle to the control reader, through the new `FileReadAt::from_file`, to the buffered data reader, and to the mapping. Direct I/O needs a descriptor of its own, so it checks that reopening the path reached the same file. Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
Since #10716, opening a file for CUDA reads on devices that read pageable memory maps the file so the device can copy cached pages directly. A failed `mmap` failed the whole open, although the mapping only speeds up reads that staging serves as well. Files on filesystems without shared mappings, such as some FUSE mounts, could therefore no longer be read at all on those devices. Fall back to staged reads when the file cannot be mapped. Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
`CudaOpenOptions::open_path` opened the path three times: for the footer and zone maps, for the pooled data reader, and, since #10716, for the mapping the device copies cached pages from. A file atomically replaced at that path between the opens, as writers commonly publish files, could therefore pair one file's footer with another's segments, or serve resident ranges from one file's mapping and other ranges from the other file. Files with compatible layouts then decoded wrong data without any error. Open the file once and give that handle to the control reader, through the new `FileReadAt::from_file`, to the buffered data reader, and to the mapping. Direct I/O needs a descriptor of its own, so it checks that reopening the path reached the same file. Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
Since #10716, opening a file for CUDA reads on devices that read pageable memory maps the file so the device can copy cached pages directly. A failed `mmap` failed the whole open, although the mapping only speeds up reads that staging serves as well. Files on filesystems without shared mappings, such as some FUSE mounts, could therefore no longer be read at all on those devices. Fall back to staged reads when the file cannot be mapped. Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
`CudaOpenOptions::open_path` opened the path three times: for the footer and zone maps, for the pooled data reader, and, since #10716, for the mapping the device copies cached pages from. A file atomically replaced at that path between the opens, as writers commonly publish files, could therefore pair one file's footer with another's segments, or serve resident ranges from one file's mapping and other ranges from the other file. Files with compatible layouts then decoded wrong data without any error. Open the file once and give that handle to the control reader, through the new `FileReadAt::from_file`, to the buffered data reader, and to the mapping. Direct I/O needs a descriptor of its own, so it checks that reopening the path reached the same file. Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
Problem
Cached file pages still pass through pinned staging buffers before reaching the GPU.
Solution
On supported Linux devices, copy resident mapped-file ranges directly with a CUDA kernel; retain staged reads for other cases.
Assessment
Hardware-dependent fast path with mapping lifetime and cancellation guards. Keeps 4 MiB chunks, so it is independent of #10715. Based on #10726.