chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

[Bugfix] For cp: Fixed hang problem in prefix cache and kvcache support fp8 in-seq-split mode (#19656)

Browse Source 
Co-authored-by: vincent <vincent@vincentdeMacBook-Pro.local>
This commit is contained in:
Baidu-AIAK
2026-03-04 11:19:46 +08:00
committed by GitHub
parent 88290690f0
commit 6851613b93
3 changed files with 4 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
docs/basic_usage/deepseek_v32.md
View File

@@ -316,8 +316,7 @@ The first mode can be enabled by `--nsa-prefill-cp-mode in-seq-split`. This mode
Note that in sequence splitting mode has the following restrictions:
- The batch size is restricted to 1 for prefill batches
- Multi-node/PD disaggregation is still not supported
- `moe_dense_tp_size=1`, `kv_cache_dtype = "bf16"`, `moe_a2a_backend = "deepep"`
- `moe_dense_tp_size=1`, `moe_a2a_backend = "deepep"`
- To ensure `cp_size > 1`, the passed in `tp_size` must be larger than `dp_size`
For more details, please refer to PR https://github.com/sgl-project/sglang/pull/12065.