chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

DOC update nemo-skills in docs (#14555)

Browse Source 
Signed-off-by: George Armstrong <georgea@nvidia.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
This commit is contained in:
George Armstrong
2025-12-06 19:03:08 -08:00
committed by GitHub
parent 32a32cf7d2
commit 91c9c14c28
1 changed files with 17 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files

50
docs/basic_usage/deepseek_v32.md
View File

@@ -1,10 +1,10 @@
# DeepSeek V3.2 Usage
DeepSeek-V3.2 model families equips DeepSeek-V3.1-Terminus with DeepSeek Sparse Attention (DSA) through continued training. With DSA, a fine-grained sparse attention mechanism powered by a lightning indexer, DeepSeek-V3.2 achieves efficiency improvements in long-context scenarios.
DeepSeek-V3.2 model family equips DeepSeek-V3.1-Terminus with DeepSeek Sparse Attention (DSA) through continued training. With DSA, a fine-grained sparse attention mechanism powered by a lightning indexer, DeepSeek-V3.2 achieves efficiency improvements in long-context scenarios.
For reporting issues or tracking upcoming features, please refer to this [Roadmap](https://github.com/sgl-project/sglang/issues/11060).
Note: This document is originally written for the usage of [DeepSeek-V3.2-Exp](https://huggingface.co/deepseek-ai/DeepSeek-V3.2-Exp) model. The usage of [DeepSeek-V3.2](https://huggingface.co/deepseek-ai/DeepSeek-V3.2) or [DeepSeek-V3.2-Speciale](https://huggingface.co/deepseek-ai/DeepSeek-V3.2-Speciale) is the same as DeepSeek-V3.2-Exp except the tool call parser.
Note: This document is originally written for the usage of [DeepSeek-V3.2-Exp](https://huggingface.co/deepseek-ai/DeepSeek-V3.2-Exp) model. The usage of [DeepSeek-V3.2](https://huggingface.co/deepseek-ai/DeepSeek-V3.2) or [DeepSeek-V3.2-Speciale](https://huggingface.co/deepseek-ai/DeepSeek-V3.2-Speciale) is the same as DeepSeek-V3.2-Exp except for the tool call parser.
## Installation
@@ -197,34 +197,18 @@ Prepare the environment by installing NeMo-Skills in the docker or your own virt
pip install git+https://github.com/NVIDIA/NeMo-Skills.git --ignore-installed blinker
```
Nemo Skill can't enable thinking method from client side, so we need some hardcoding before launching server:
**For `DeepSeek-V3.2-Exp`**:
Modify the [`jinja chat_template`](https://huggingface.co/deepseek-ai/DeepSeek-V3.2-Exp/blob/main/tokenizer_config.json#L34) by replacing
Then launch the SGLang server:
```
{% set thinking = false %}
```
with
```
{% set thinking = true %}
```
and save it to `chat_template_thinking.jinja`.
Then launch the SGLang server with the modified chat-template file:
```
python -m sglang.launch_server --model deepseek-ai/DeepSeek-V3.2-Exp --tp 8 --dp 8 --enable-dp-attention --chat-template chat_template_thinking.jinja
python -m sglang.launch_server --model deepseek-ai/DeepSeek-V3.2-Exp --tp 8 --dp 8 --enable-dp-attention
```
**For `DeepSeek-V3.2` and `DeepSeek-V3.2-Speciale`**:
Hardcode the thinking mode to be `thinking` in `_apply_jinja_template` function [here](https://github.com/sgl-project/sglang/blob/7c38eca1e4a704bf09fe6b52ea040a41d3cfc55d/python/sglang/srt/entrypoints/openai/serving_chat.py#L286), then launch the server as usual:
```
python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-V3.2 --trust-remote-code --tp-size 8 --dp-size 8 --enable-dp-attention --tool-call-parser deepseekv32 --reasoning-parser deepseek-v3
```
After hardcoding, run the following script to evaluate AIME 2025:
Run the following script to evaluate AIME 2025:
```
#! /bin/bash
export NEMO_SKILLS_DISABLE_UNCOMMITTED_CHANGES_CHECK=1
@@ -243,38 +227,38 @@ ns eval \
--model=$MODEL \
--server_address=http://localhost:${PORT}/v1 \
--output_dir=nemo_skills_aime25_${MODEL_NAME}_output_${BACKEND}_$(date +%Y%m%d_%H%M%S) \
++chat_template_kwargs.thinking=true \
++inference.temperature=1.0 \
++inference.top_p=0.95 \
++inference.tokens_to_generate=64000
# ++inference.tokens_to_generate=120000 for Speciale model
```
Test results:
Test results (8*B200):
DeepSeek-V3.2-Exp：
| evaluation_mode | num_entries | avg_tokens | gen_seconds | symbolic_correct | no_answer |
| evaluation_mode | num_entries | avg_tokens | gen_seconds | symbolic_correct | no_answer |
|--------------------|-------------|------------|-------------|-----------------------|-----------|
| pass@1[avg-of-4] | 30 | 14410 | 1758 | 85.83% ± 4.19% | 0.00% |
| majority@4 | 30 | 14410 | 1758 | 90.00% | 0.00% |
| pass@4 | 30 | 14410 | 1758 | 93.33% | 0.00% |
| pass@1[avg-of-4] | 30 | 15040 | 1673 | 87.50% ± 1.67% | 0.00% |
| majority@4 | 30 | 15040 | 1673 | 90.00% | 0.00% |
| pass@4 | 30 | 15040 | 1673 | 90.00% | 0.00% |
Note that the result of problem#3 with id `aime25-2` is marked as false by nemo-skills but is actually correct because nemo-skills fails to match predicted_answer `016` with expected_answer `16`. If we add 1/30 = 3.33% to the results, the pass@1[avg-of-4] results match with official reference. (89.3%)
DeepSeek-V3.2:
| evaluation_mode | num_entries | avg_tokens | gen_seconds | symbolic_correct | no_answer |
|--------------------|-------------|------------|-------------|-----------------------|-----------|
| pass@1[avg-of-4] | 30 | 13311 | 1630 | 90.83% ± 4.19% | 0.00% |
| majority@4 | 30 | 13311 | 1630 | 90.00% | 0.00% |
| pass@4 | 30 | 13311 | 1630 | 96.67% | 0.00% |
| pass@1[avg-of-4] | 30 | 13550 | 1632 | 92.50% ± 1.67% | 0.00% |
| majority@4 | 30 | 13550 | 1632 | 94.71% | 0.00% |
| pass@4 | 30 | 13550 | 1632 | 96.67% | 0.00% |
DeepSeek-V3.2-Speciale:
| evaluation_mode | num_entries | avg_tokens | gen_seconds | symbolic_correct | no_answer |
|--------------------|-------------|------------|-------------|-----------------------|-----------|
| pass@1[avg-of-4] | 30 | 21210 | 3018 | 95.00% ± 3.33% | 0.00% |
| majority@4 | 30 | 21210 | 3018 | 96.67% | 0.00% |
| pass@4 | 30 | 21210 | 3018 | 100.00% | 0.00% |
| pass@1[avg-of-4] | 30 | 24155 | 3583 | 95.00% ± 1.92% | 0.00% |
| majority@4 | 30 | 24155 | 3583 | 95.83% | 0.00% |
| pass@4 | 30 | 24155 | 3583 | 100.00% | 0.00% |