chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

[DeepSeek] Update tests and document for DeepSeek V3.2 NVFP4 checkpoint (#17657)

Browse Source
This commit is contained in:
Baizhou Zhang
2026-01-27 22:10:57 +08:00
committed by GitHub
parent d578b41bad
commit 1d942e4eef
3 changed files with 88 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
docs/basic_usage/deepseek_v32.md
View File

@@ -125,6 +125,14 @@ python3 -m sglang.launch_server \
--reasoning-parser deepseek-v3
```
## NVFP4 Checkpoint
To launch deepseek v3.2 [NVFP4 checkpoint](https://huggingface.co/nvidia/DeepSeek-V3.2-NVFP4) on Blackwell devices, the user needs to specify the quantization method as `modelopt_fp4`, and moe runner backend as one of `flashinfer_trtllm`(recommended), `flashinfer_cutlass` and `flashinfer_cutedsl`. Any other usage (parallelism, reasoning parser, ...) is the same as FP8 checkpoint.
An example launching command can be:
```bash
python -m sglang.launch_server --model nvidia/DeepSeek-V3.2-NVFP4 --tp 4 --quantization modelopt_fp4 --moe-runner-backend flashinfer_trtllm --tool-call-parser deepseekv32 --reasoning-parser deepseek-v3
```
## PD Disaggregation