[Doc] Add tip on how to use Spec V2 (#15455)

This commit is contained in:
b8zhong
2026-01-16 05:30:18 +08:00
committed by GitHub
parent 7dde3438e2
commit 3d72944fb8
3 changed files with 27 additions and 7 deletions
+18
View File
@@ -59,6 +59,24 @@ python -m sglang.launch_server ... --tool-server ip-1:port-1,ip-2:port-2
```
The URLs should be MCP SSE servers that expose server information and well-documented tools. These tools are added to the system prompt so the model can use them.
## Speculative Decoding
SGLang supports speculative decoding for GPT-OSS models using EAGLE3 algorithm. This can significantly improve decoding speed, especially for small batch sizes.
**Usage**:
Add `--speculative-algorithm EAGLE3` along with the draft model path.
```bash
python3 -m sglang.launch_server \
--model-path openai/gpt-oss-120b \
--speculative-algorithm EAGLE3 \
--speculative-draft-model-path lmsys/EAGLE3-gpt-oss-120b-bf16 \
--tp 2
```
```{tip}
To enable the experimental overlap scheduler for EAGLE3 speculative decoding, set the environment variable `SGLANG_ENABLE_SPEC_V2=1`. This can improve performance by enabling overlap scheduling between draft and verification stages.
```
### Quick Demo
```python