chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[NPU] update feature supported on ascend NPU (#16915)

Browse Source
This commit is contained in:
Hexq0210
2026-01-12 11:47:58 +08:00
committed by GitHub
parent a1c48943d7
commit c581b5ed79
1 changed files with 18 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/platforms/ascend_npu_support_features.md
+18 -18
View File
@@ -202,23 +202,23 @@ click [Server Arguments](https://docs.sglang.io/advanced_features/server_argumen
## Speculative decoding
| Argument | Defaults | Options | A2 | A3 |
|------------------------------------------------------------------|-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------:|:----------------------------------------:|
| `--speculative-algorithm` | `None` | `EAGLE3`,<br/> `NEXTN` | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-draft-model-path`<br/>`--speculative-draft-model` | `None` | Type: str | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-draft-model-`<br/>`revision` | `None` | Type: str | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-draft-load-format` | `None` | `auto`,<br/>`pt`,<br/>`safetensors`,<br/>`npcache`,<br/>`dummy`,<br/>`sharded_state`,<br/>`gguf`,<br/>`bitsandbytes`,<br/>`layered`,<br/>`flash_rl`,<br/>`remote`,<br/>`remote_instance`,<br/>`fastsafetensors`,<br/>`private` | **<span style="color: red;">×</span>** | **<span style="color: red;">×</span>** |
| `--speculative-num-steps` | `None` | Type: int | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-eagle-topk` | `None` | Type: int | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-num-draft-tokens` | `None` | Type: int | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-accept-`<br/>`threshold-single` | `1.0` | Type: float | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-accept-`<br/>`threshold-acc` | `1.0` | Type: float | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-token-map` | `None` | Type: str | **<span style="color: red;">×</span>** | **<span style="color: red;">×</span>** |
| `--speculative-attention-`<br/>`mode` | `prefill` | `prefill`,<br/> `decode` | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-moe-runner-`<br/>`backend` | `None` | `auto`,<br/>`deep_gemm`,<br/>`triton`,<br/>`triton_kernel`,<br/>`flashinfer_trtllm`,<br/>`flashinfer_cutlass`,<br/>`flashinfer_mxfp4`,<br/>`flashinfer_cutedsl`,<br/>`cutlass` | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-moe-a2a-`<br/>`backend` | `None` | `none`,<br/> `deepep`,<br/> `mooncake`,<br/> `ascend_fuseep` | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-draft-attention-backend` | `None` | Type: str | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-draft-model-quantization` | `None` | `awq`, <br/>`fp8`, <br/>`gptq`, <br/> `marlin`, <br/> `gptq_marlin`, <br/> `awq_marlin`, <br/> `bitsandbytes`, <br/> `gguf`, <br/> `modelopt`, <br/> `modelopt_fp8`, <br/> `modelopt_fp4`, <br/> `petit_nvfp4`, <br/> `w8a8_int8`, <br/> `w8a8_fp8`, <br/> `moe_wna16`, <br/> `qoq`, <br/> `w4afp8`, <br/> `mxfp4`, <br/> `auto-round`, <br/> `compressed-tensors`, <br/> `modelslim`, <br/> `unquant` | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| Argument | Defaults | Options | A2 | A3 |
|------------------------------------------------------------------|-----------|--------------------------|:----------------------------------------:|:----------------------------------------:|
| `--speculative-algorithm` | `None` | `EAGLE3`,<br/> `NEXTN` | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-draft-model-path`<br/>`--speculative-draft-model` | `None` | Type: str | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-draft-model-`<br/>`revision` | `None` | Type: str | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-draft-load-format` | `None` | `auto` | **<span style="color: red;">×</span>** | **<span style="color: red;">×</span>** |
| `--speculative-num-steps` | `None` | Type: int | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-eagle-topk` | `None` | Type: int | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-num-draft-tokens` | `None` | Type: int | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-accept-`<br/>`threshold-single` | `1.0` | Type: float | **<span style="color: red;">×</span>** | **<span style="color: red;">×</span>** |
| `--speculative-accept-`<br/>`threshold-acc` | `1.0` | Type: float | **<span style="color: red;">×</span>** | **<span style="color: red;">×</span>** |
| `--speculative-token-map` | `None` | Type: str | **<span style="color: red;">×</span>** | **<span style="color: red;">×</span>** |
| `--speculative-attention-`<br/>`mode` | `prefill` | `prefill`,<br/> `decode` | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-moe-runner-`<br/>`backend` | `None` | `auto` | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-moe-a2a-`<br/>`backend` | `None` | `ascend_fuseep` | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-draft-attention-backend` | `None` | `ascend` | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--speculative-draft-model-quantization` | `None` | `unquant` | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
## Ngram speculative decoding
@@ -278,7 +278,7 @@ click [Server Arguments](https://docs.sglang.io/advanced_features/server_argumen
| `--radix-eviction-policy` | `lru` | `lru`, `lfu` | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--hicache-io-backend` | `kernel` | `kernel_ascend`,<br/>`direct` | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--hicache-mem-layout` | `layer_first` | `page_first_direct`,<br/> `page_first_kv_split` | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--hicache-storage-`<br/>`backend` | `None` | `file` | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
| `--hicache-storage-`<br/>`backend` | `None` | `file` | **<span style="color: red;">×</span>** | **<span style="color: red;">×</span>** |
| `--hicache-storage-`<br/>`prefetch-policy` | `best_effort` | `best_effort`,<br/> `wait_complete`,<br/> `timeout` | **<span style="color: red;">×</span>** | **<span style="color: red;">×</span>** |
| `--hicache-storage-`<br/>`backend-extra-config` | `None` | Type: str | **<span style="color: red;">×</span>** | **<span style="color: red;">×</span>** |