(1/n)support context parallel with deepseekv3.2-DSA (#12065)

This commit is contained in:
lixiaolx
2025-11-17 12:12:25 +08:00
committed by GitHub
parent 7e626d12b7
commit d368c7451a
17 changed files with 1247 additions and 54 deletions

View File

@@ -142,3 +142,23 @@ The mean accuracy over 8 runs shows 0.797, which matches the number 79.9 in offi
Repeat: 8, mean: 0.797
Scores: ['0.808', '0.798', '0.808', '0.798', '0.783', '0.788', '0.803', '0.793']
```
## DSA long sequence context parallel optimization(experimental)
Accuracy benchmark on long context can be tested on GPQA-diamond dataset with long output tokens and thinking enabled:
Example usage:
```bash
# Launch with EP + DP
python -m sglang.launch_server --model deepseek-ai/DeepSeek-V3.2-Exp --tp 8 --ep 8 --dp 2 --enable-dp-attention --enable-nsa-prefill-context-parallel --max-running-requests 32
```
### Context-parallel Tips
`CP_size` reuses `atten_tp_size`, which is equal to `TP_size` / `DP_size`.
Some features are still not supported at present.
- **Multi-batch prefill**: Currently, only single-request processing is supported during the prefill process.
- **disaggregation**: P/D disaggregation.
- **Cross-machine support**: - Currently only tested on a single machine (TP=8,EP=8).
- **Other Args**: Currently only supports moe_dense_tp_size=1, kv_cache_dtype = "bf16", moe_a2a_backend = "deepep",
- **DP_size**: `CP_size` reuses `atten_tp_size`, which is equal to `TP_size` / `DP_size`. For the cp function to work correctly, `TP_size` must be divisible by `DP_size`, and TP_size / DP_size > 1 (to ensure CP_size > 1).
- **Detailed design reference**: https://github.com/sgl-project/sglang/pull/12065