Repository navigation
Backport generator changes to 12.9.x - #2110
Merged
Merged
Conversation
github-actions
Bot
added
the
cuda.bindings
kkraus14
approved these changes
May 19, 2026
kkraus14
enabled auto-merge (squash)
May 19, 2026 20:15
yyihuang
added a commit
to flashinfer-ai/flashinfer
that referenced
this pull request
Oct 28, 2026
…test expectations (#7632) ## Summary This PR fixes three test failures from the scheduled nightly runs of the test suite. It changes no kernel sources: - On toolkits older than CUDA 13.4, NVFP4 context attention programs now fail closed instead of failing in ptxas. - An SVDQuant tail-split test now checks the actual schedule rule. - The warp-decode PDL test now reads graph edges without a cuda-bindings use-after-free. ## Changes **NVFP4 context programs require CUDA 13.4** (`flashinfer/cake_fmha.py`, `tests/attention/test_cake_fmha_context_programs.py`) - The NVFP4 programs under `csrc/cake_fmha_context_nvfp4/` (Q511 and Q511_SCALE_PTR, SM100A/SM103A) dequantize E2M1 pages with the PTX ISA 9.4 `mul.rn.satfinite.e4m3x4.e2m1x4.e4m3x4`. Only CUDA 13.4+ supports this instruction. With CUDA 12.9 and 13.0, ptxas rejects it (`Illegal modifier '.satfinite' for instruction 'mul'`). This failed `test_cake_context_quantized_programs_match_reference[case2-Q511]`, `[case3-Q511_SCALE_PTR]` and the six `test_cake_context_programs_match_the_jit_route_bitwise[nvfp4_*]` rows on B200, GB200 and GB300. CUDA 13.4 passed. - Add `CONTEXT_NVFP4_PROGRAMS_MIN_CUDA = "13.4"` and `context_nvfp4_programs_unavailable()`, which returns a reason string or `None` via `is_cuda_version_at_least`. - On an older toolkit, `get_cake_fmha_context_module` routes an NVFP4 program route to compat_v1 with a `RuntimeWarning` ("failed closed to compat_v1"), as the NVFP4 JIT route already does. `_load_cake_fmha_context_program` raises a `RuntimeError` naming the program and the reason instead of launching a build that ptxas rejects. `flashinfer/jit/cake_dsv4.py` already requires CUDA 13.4 for the same instruction in the DSv4 NVFP4 decode kernels. - Tests: on such toolkits, `_skip_without_nvfp4_programs()` skips the NVFP4 rows and states the reason. The new `test_cake_context_nvfp4_programs_fail_closed_without_their_toolkit` monkeypatches the check. It asserts that the loader raises and that the route falls back to compat_v1 with the warning. **SVDQuant tail-split routing test** (`tests/gemm/test_cake_nvfp4_svdquant_gemm_backend.py`) - `test_tailsplit_rows_split_on_this_device` required every `pair_tailsplit` row to split its tail (`split == 2 and full < units`) on 148/152-SM devices. Since #7516, the row M 6889, N 12288, K 3072, rank 32, bias dispatches there on sm_100a. - On 148 SMs (74 CTA pairs) the row has 1296 pair tiles (27 × 48): 17 full waves plus 38 tail tiles. - 2 × 38 = 76 half units exceed 74 pairs, so `tailsplit_units` keeps whole tiles (`tail_split = 1`). - B200 therefore failed with `AssertionError: ('M6889-N12288-K3072-r32-bias', 148)` under CUDA 13.0 and 13.4. - The test now asserts the schedule rule. A partial last wave must exist (`tail > 0`). If `2 * tail <= pairs`, the tail becomes `2 * tail` half units; otherwise all tiles stay whole. - `test_tailsplit_units_schedule` adds `units(m=6889, n=12288, sm_count=148) == (1296, 1296, 1296, 1)` and `units(m=6889, n=12288, sm_count=152) == (1296, 1300, 1292, 2)`. - There is no loader or kernel change, and the #7516 dispatch stays as is. **Warp-decode PDL graph edges** (`tests/moe/test_cake_warp_decode_pdl_gpu.py`) - `test_first_kernel_launch_dependency[direct|route_pack|fused_route_pack]` read captured-graph edges with `cuda.bindings.driver.cuGraphGetEdges` (`cuGraphGetEdges_v2` in the CUDA 12 bindings). - cuda-bindings before 12.9.7 and 13.3.0 return `CUgraphEdgeData` wrappers backed by a buffer that is freed when the call returns (NVIDIA/cuda-python#1804, fixed in NVIDIA/cuda-python#2110). - In the CUDA 13.0 environment (cuda-bindings 13.0.3), the test saw edge types 255, 250, 76, 178, 40, 57, ... instead of 1 (programmatic) or 0 (default). - The 12.9.7 and 13.4.2 bindings passed. - The new `_graph_edges(graph_handle)` calls `cuGraphGetEdges_v2` from `libcuda.so.1` via ctypes. It uses a `CUgraphEdgeData` Structure (`from_port`, `to_port`, `type`, `reserved[5]`). - `_kernel_graph` still uses cuda-bindings for node enumeration, node types and kernel names, which the bug does not affect. ## Validation Validation is in flight and will be summarized here once the GPU matrix (B200, GB200, GB300 with the CUDA 12.9, 13.0 and 13.4 environments) on the three test files and the public CI run have completed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **Compatibility** * NVFP4 context programs require CUDA 13.4 or newer. On older toolkits, loading these programs reports the version requirement, and module selection warns before falling back to a compatible implementation. This makes the toolkit limitation clearer and allows supported workloads to use the fallback where available. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Avery Huang <averyh@nvidia.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rolls up backports for 3 generator changes.