Repository navigation
Conversation
…ine are used consecutively. (deepseek-ai#580)
* Add hybrid EP token capacity validation * Pad hybrid EP token capacity to combine chunk
* hybrid_ep: fix scan permute-fusion local-experts sum array sizing In the permute-fusion path of `scan`, `thread_local_all_block_local_experts_sum` was declared with `NUM_OF_EXPERTS_PER_RANK - 1` slots, but the load loop just below writes index `i` for `i in [0, NUM_OF_EXPERTS_PER_RANK)`. When `i == NUM_OF_EXPERTS_PER_RANK - 1` this overflows the stack array. Bump the size to `NUM_OF_EXPERTS_PER_RANK` so all written slots are valid. * nixl: coalesce dispatch prob put with dest-major source layout The dispatch N2N warp issued one nixlPut per token for the prob slice because the source layout `[num_tokens][num_of_nodes][prob_per_token]` interleaved per-destination prob slices on token boundaries, so per-destination prob bytes weren't contiguous in memory. Restripe prob into `[num_of_nodes][max_tokens_per_rank][prob_per_token]` in `dispatch_preprocess` (replacing the existing `cudaMemcpyAsync`), so the N2N warp issues one coalesced nixlPut per chunk in the dense path and one per run in the sparse path. The G2S warp's local-node prob branch is updated to address the dest-major layout (`node_rank * MAX_TOKENS` base, `prob_per_token` per-token stride). DOCA path is preserved via `#ifdef USE_NIXL` guards; receive-side layout is unchanged. Adds `MAX_NUM_OF_TOKENS_PER_RANK` as a template parameter on the NIXL N2N warp group.
* fix(build): inherit PyTorch C++ standard for hybrid EP * test(build): keep standard guard Python 3.8 compatible
…ek-ai#724) * [HybridEP] Support configurable RoCE settings in DOCA backend * Fix RoCE configuration edge cases
…pseek-ai#750) * fix(hybrid_ep): add DMA-BUF registration fallback * fix(hybrid_ep): harden DMA-BUF registration fallback * refactor(hybrid_ep): keep GPU registration state in coordinator
Bring the Paddle-compatible HybridEP branch up to upstream 4e890fd ([HybridEP] Fix the fused combine on a non-blocking dispatch, deepseek-ai#766) and re-apply the PaddlePaddle compat layer on top of the refactored upstream code. Notable resolutions: - Upstream moved the NVL handle exchange into NVLCoordinator; the Paddle port (paddle.distributed, `world_size` attribute, explicit host->device copy) moved with it, now sized from the tensor instead of a hardcoded handle count, which upstream changed from 4/3 to 5/4. - Kept upstream's dense (int16 topk_idx) routing and its 16B-alignment fallback, with the collective switched to paddle.distributed.stream. - Kept the Paddle JIT file lock and threaded upstream's new permute-fusion / token-drop build flags through it. - indices_to_map keeps upstream's masked scatter_add_, which handles the out-of-range topk_idx that ragged dispatch produces; Paddle's compat scatter_add_ is put_along_axis_(reduce='add') underneath, so PFCCLab#16's intent is preserved and that commit became empty. - Test suite follows upstream, with the Paddle-only adaptations kept (enable_compat, fleet launch, bf16 allclose, uint8 index_select, per-node local rank, Kineto skip). Verified on GB300 (sm_103): deep_ep_cpp and hybrid_ep_cpp both build; tests/test_hybrid_ep.py passes all 12 correctness checks (BF16 + FP8, sparse + dense routing, fused + non-fused) on 1 node/4 GPUs, 2 nodes/8 GPUs and 3 nodes/12 GPUs, the latter two over RoCE via the DOCA path. Co-Authored-By: Claude Code <noreply@anthropic.com>
Difers
force-pushed
the
hybrid-ep-paddle
branch
from
November 9, 2026 09:00
78429c4 to
e8f6bf5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Update Hybrid EP