From 91c9c14c282b6268359927aa12fdede32e8b2d02 Mon Sep 17 00:00:00 2001 From: George Armstrong Date: Sat, 6 Dec 2025 19:03:08 -0800 Subject: [PATCH] DOC update nemo-skills in docs (#14555) Signed-off-by: George Armstrong Co-authored-by: Baizhou Zhang --- docs/basic_usage/deepseek_v32.md | 50 +++++++++++--------------------- 1 file changed, 17 insertions(+), 33 deletions(-) diff --git a/docs/basic_usage/deepseek_v32.md b/docs/basic_usage/deepseek_v32.md index abaaaf741..99c453165 100644 --- a/docs/basic_usage/deepseek_v32.md +++ b/docs/basic_usage/deepseek_v32.md @@ -1,10 +1,10 @@ # DeepSeek V3.2 Usage -DeepSeek-V3.2 model families equips DeepSeek-V3.1-Terminus with DeepSeek Sparse Attention (DSA) through continued training. With DSA, a fine-grained sparse attention mechanism powered by a lightning indexer, DeepSeek-V3.2 achieves efficiency improvements in long-context scenarios. +DeepSeek-V3.2 model family equips DeepSeek-V3.1-Terminus with DeepSeek Sparse Attention (DSA) through continued training. With DSA, a fine-grained sparse attention mechanism powered by a lightning indexer, DeepSeek-V3.2 achieves efficiency improvements in long-context scenarios. For reporting issues or tracking upcoming features, please refer to this [Roadmap](https://github.com/sgl-project/sglang/issues/11060). -Note: This document is originally written for the usage of [DeepSeek-V3.2-Exp](https://huggingface.co/deepseek-ai/DeepSeek-V3.2-Exp) model. The usage of [DeepSeek-V3.2](https://huggingface.co/deepseek-ai/DeepSeek-V3.2) or [DeepSeek-V3.2-Speciale](https://huggingface.co/deepseek-ai/DeepSeek-V3.2-Speciale) is the same as DeepSeek-V3.2-Exp except the tool call parser. +Note: This document is originally written for the usage of [DeepSeek-V3.2-Exp](https://huggingface.co/deepseek-ai/DeepSeek-V3.2-Exp) model. The usage of [DeepSeek-V3.2](https://huggingface.co/deepseek-ai/DeepSeek-V3.2) or [DeepSeek-V3.2-Speciale](https://huggingface.co/deepseek-ai/DeepSeek-V3.2-Speciale) is the same as DeepSeek-V3.2-Exp except for the tool call parser. ## Installation @@ -197,34 +197,18 @@ Prepare the environment by installing NeMo-Skills in the docker or your own virt pip install git+https://github.com/NVIDIA/NeMo-Skills.git --ignore-installed blinker ``` -Nemo Skill can't enable thinking method from client side, so we need some hardcoding before launching server: - -**For `DeepSeek-V3.2-Exp`**: - -Modify the [`jinja chat_template`](https://huggingface.co/deepseek-ai/DeepSeek-V3.2-Exp/blob/main/tokenizer_config.json#L34) by replacing - +Then launch the SGLang server: ``` -{% set thinking = false %} -``` -with -``` -{% set thinking = true %} -``` -and save it to `chat_template_thinking.jinja`. - -Then launch the SGLang server with the modified chat-template file: -``` -python -m sglang.launch_server --model deepseek-ai/DeepSeek-V3.2-Exp --tp 8 --dp 8 --enable-dp-attention --chat-template chat_template_thinking.jinja +python -m sglang.launch_server --model deepseek-ai/DeepSeek-V3.2-Exp --tp 8 --dp 8 --enable-dp-attention ``` **For `DeepSeek-V3.2` and `DeepSeek-V3.2-Speciale`**: -Hardcode the thinking mode to be `thinking` in `_apply_jinja_template` function [here](https://github.com/sgl-project/sglang/blob/7c38eca1e4a704bf09fe6b52ea040a41d3cfc55d/python/sglang/srt/entrypoints/openai/serving_chat.py#L286), then launch the server as usual: ``` python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-V3.2 --trust-remote-code --tp-size 8 --dp-size 8 --enable-dp-attention --tool-call-parser deepseekv32 --reasoning-parser deepseek-v3 ``` -After hardcoding, run the following script to evaluate AIME 2025: +Run the following script to evaluate AIME 2025: ``` #! /bin/bash export NEMO_SKILLS_DISABLE_UNCOMMITTED_CHANGES_CHECK=1 @@ -243,38 +227,38 @@ ns eval \ --model=$MODEL \ --server_address=http://localhost:${PORT}/v1 \ --output_dir=nemo_skills_aime25_${MODEL_NAME}_output_${BACKEND}_$(date +%Y%m%d_%H%M%S) \ + ++chat_template_kwargs.thinking=true \ ++inference.temperature=1.0 \ ++inference.top_p=0.95 \ ++inference.tokens_to_generate=64000 # ++inference.tokens_to_generate=120000 for Speciale model ``` -Test results: +Test results (8*B200): DeepSeek-V3.2-Exp： -| evaluation_mode | num_entries | avg_tokens | gen_seconds | symbolic_correct | no_answer | +| evaluation_mode | num_entries | avg_tokens | gen_seconds | symbolic_correct | no_answer | |--------------------|-------------|------------|-------------|-----------------------|-----------| -| pass@1[avg-of-4] | 30 | 14410 | 1758 | 85.83% ± 4.19% | 0.00% | -| majority@4 | 30 | 14410 | 1758 | 90.00% | 0.00% | -| pass@4 | 30 | 14410 | 1758 | 93.33% | 0.00% | +| pass@1[avg-of-4] | 30 | 15040 | 1673 | 87.50% ± 1.67% | 0.00% | +| majority@4 | 30 | 15040 | 1673 | 90.00% | 0.00% | +| pass@4 | 30 | 15040 | 1673 | 90.00% | 0.00% | -Note that the result of problem#3 with id `aime25-2` is marked as false by nemo-skills but is actually correct because nemo-skills fails to match predicted_answer `016` with expected_answer `16`. If we add 1/30 = 3.33% to the results, the pass@1[avg-of-4] results match with official reference. (89.3%) DeepSeek-V3.2: | evaluation_mode | num_entries | avg_tokens | gen_seconds | symbolic_correct | no_answer | |--------------------|-------------|------------|-------------|-----------------------|-----------| -| pass@1[avg-of-4] | 30 | 13311 | 1630 | 90.83% ± 4.19% | 0.00% | -| majority@4 | 30 | 13311 | 1630 | 90.00% | 0.00% | -| pass@4 | 30 | 13311 | 1630 | 96.67% | 0.00% | +| pass@1[avg-of-4] | 30 | 13550 | 1632 | 92.50% ± 1.67% | 0.00% | +| majority@4 | 30 | 13550 | 1632 | 94.71% | 0.00% | +| pass@4 | 30 | 13550 | 1632 | 96.67% | 0.00% | DeepSeek-V3.2-Speciale: | evaluation_mode | num_entries | avg_tokens | gen_seconds | symbolic_correct | no_answer | |--------------------|-------------|------------|-------------|-----------------------|-----------| -| pass@1[avg-of-4] | 30 | 21210 | 3018 | 95.00% ± 3.33% | 0.00% | -| majority@4 | 30 | 21210 | 3018 | 96.67% | 0.00% | -| pass@4 | 30 | 21210 | 3018 | 100.00% | 0.00% | +| pass@1[avg-of-4] | 30 | 24155 | 3583 | 95.00% ± 1.92% | 0.00% | +| majority@4 | 30 | 24155 | 3583 | 95.83% | 0.00% | +| pass@4 | 30 | 24155 | 3583 | 100.00% | 0.00% |