chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[V3.2] Change default CP token split method to --round-robin-split (#18613)

Browse Source
This commit is contained in:
Baizhou Zhang
2026-02-11 20:14:35 +08:00
committed by GitHub
parent 4f7422f7ba
commit 947927bdb5
3 changed files with 5 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/basic_usage/deepseek_v32.md
View File

@@ -306,7 +306,7 @@ DeepSeek-V3.2-Speciale:
For context parallel in DeepSeek V3.2 model, we provide two different modes of splitting tokens, which can be controlled with argument `--nsa-prefill-cp-mode`.
### In sequence splitting (default setting)
### In sequence splitting
The first mode can be enabled by `--nsa-prefill-cp-mode in-seq-split`. This mode implements context parallel for DSA by splitting the sequence uniformly between context parallel ranks. At attention stage, each cp rank computes the indexer results of sharded sequence, and collects the whole kv cache through all gather operator.
@@ -326,7 +326,7 @@ Example:
python -m sglang.launch_server --model deepseek-ai/DeepSeek-V3.2-Exp --tp 8 --ep 8 --dp 2 --enable-dp-attention --enable-nsa-prefill-context-parallel --nsa-prefill-cp-mode in-seq-split --max-running-requests 32
```
### Round robin splitting
### Round robin splitting (default setting)
This mode can be enabled by specifying the parameter `--nsa-prefill-cp-mode round-robin-split`, which distributes tokens across ranks based on `token_idx % cp_size`.