chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

B300 CP decouple from DeepEP (Strategy B), stage A: relax forced moe_a2a_backend + allow tp4/CP4

Browse Source 
NSA-prefill CP in-seq-split no longer hard-forces moe_a2a_backend=deepep. Only default to deepep
when the user left both moe_a2a_backend=none AND moe_runner_backend=auto; with an explicit runner
(e.g. flashinfer_trtllm) keep a2a on the standard/allgather path, validated to {deepep,flashinfer,none}.
Adapted from wxiwnd@b300-combact-fix 317acf6c99 (reconciled to our HEAD). Also relax the tp_size==8
assert to single-node tp_size in {4,8} so the GLM-5.2-NVFP4 tp4/CP4 PD target can start; CP code is
parametric in cp_size and the tp%cp / tp%(dp*cp) divisibility asserts still guard validity.

GLM-5.2 (GlmMoeDsaForCausalLM = DeepSeek v3.2 arch) inherits DeepseekV2DecoderLayer, which already
selects NSACPLayerCommunicator under CP -> no GLM-side communicator change needed (decouple stage B
N/A; Glm4MoeDecoderLayer is the unrelated GLM4 arch). Runtime FULL<->SCATTERED verify pending launch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
孙远航
2026-06-21 15:05:45 +00:00
parent 2670c6c51b
commit c616bb7095
1 changed files with 26 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

32
python/sglang/srt/server_args.py
View File

@@ -1795,13 +1795,28 @@ class ServerArgs:
"Context parallel feature is still under experiment. It has only been verified on Hopper platform."
)
if self.nsa_prefill_cp_mode == "in-seq-split":
# TODO Supports moe_dense_tp_size != 1, kv cache dtype = "fp8",moe_a2a_backend non-deepep and cross-machine operation .
# TODO Supports moe_dense_tp_size != 1, kv cache dtype = "fp8", and cross-machine operation .
self.enable_dp_attention = True
self.moe_dense_tp_size = 1
self.moe_a2a_backend = "deepep"
# B300 decouple (Strategy B): only force the historical
# deepep CP path when no MoE backend was selected
# explicitly. With an explicit runner (e.g.
# flashinfer_trtllm) keep a2a as "none"/"flashinfer"
# so CP runs on the standard/allgather path.
if (
self.moe_a2a_backend == "none"
and self.moe_runner_backend == "auto"
):
self.moe_a2a_backend = "deepep"
if self.moe_a2a_backend not in {"deepep", "flashinfer", "none"}:
raise ValueError(
"For NSA prefill CP in-seq split mode, moe_a2a_backend "
"must be one of {'deepep', 'flashinfer', 'none'}."
)
self.ep_size = self.tp_size
logger.warning(
"For in-seq split mode, we have the following restrictions: moe_dense_tp_size == 1, moe_a2a_backend == deepep, ep_size == tp_size, batch_size == 1"
"For in-seq split mode: moe_dense_tp_size == 1, moe_a2a_backend in "
"{'deepep', 'flashinfer', 'none'}, ep_size == tp_size, batch_size == 1"
)
else:
self.enable_dp_attention = True
@@ -1809,9 +1824,14 @@ class ServerArgs:
assert (
self.dp_size == 1
), "For round-robin split mode, dp attention is not supported."
assert (
self.tp_size == 8
), "Current multi-machine CP support suffers from precision issues. So context parallel only support Single machine(tp_size == 8)"
# B300 decouple: allow single-node tp4/CP4 (target PD) in
# addition to tp8/CP8. CP code is parametric in cp_size; the
# tp_size%attn_cp_size and tp_size%(dp_size*attn_cp_size)
# divisibility asserts below still guard validity.
assert self.tp_size in (4, 8), (
"Multi-machine CP has precision issues; context parallel only "
f"supports a single machine (tp_size in {{4, 8}}), got {self.tp_size}."
)
self.attn_cp_size = self.tp_size
logger.warning(