diff --git a/docs/platforms/ascend_npu_support_features.md b/docs/platforms/ascend_npu_support_features.md
index b66d2eecb..55b7360d0 100644
--- a/docs/platforms/ascend_npu_support_features.md
+++ b/docs/platforms/ascend_npu_support_features.md
@@ -4,7 +4,7 @@ This section describes the basic functions and features supported by the Ascend
questions, please [open an issue](https://github.com/sgl-project/sglang/issues).
If you want to know the meaning and usage of each parameter,
-click [Service Arguments](https://docs.sglang.io/advanced_features/server_arguments.html).
+click [Server Arguments](https://docs.sglang.io/advanced_features/server_arguments.html).
## Model and tokenizer
@@ -22,20 +22,20 @@ click [Service Arguments](https://docs.sglang.io/advanced_features/server_argume
| `--is-embedding` | `False` | bool flag (set to enable) | **√** | **√** |
| `--enable-multimodal` | `None` | bool flag (set to enable) | **√** | **√** |
| `--revision` | `None` | Type: str | **×** | **×** |
-| `--model-impl` | `None` | `auto`, `sglang`,
`transformers` | **√** | **√** |
+| `--model-impl` | `auto` | `auto`, `sglang`,
`transformers` | **√** | **√** |
## HTTP server
-| Argument | Defaults | Options | A2 | A3 |
-|-------------------------------------------------------------------|-------------|---------------------------------|:----------------------------------------:|:----------------------------------------:|
-| `--host` | `127.0.0.1` | Type: str | **√** | **√** |
-| `--port` | `30000` | Type: int | **√** | **√** |
-| `--skip-server-warmup` | `False` | bool flag
(set to enable) | **√** | **√** |
-| `--warmups` | `None` | Type: str | **√** | **√** |
-| `--nccl-port` | `None` | Type: int | **√** | **√** |
-| `--fastapi-root-path` | `None` | Type: int | **×** | **×** |
-| `--grpc-mode` | `None` | Type: int | **×** | **×** |
-| `--checkpoint-engine-`
`wait-weights-`
`before-ready` | FALSE | bool flag
(set to enable) | **×** | **×** |
+| Argument | Defaults | Options | A2 | A3 |
+|-------------------------------------------------------------------|-------------|--------------------------------|:----------------------------------------:|:----------------------------------------:|
+| `--host` | `127.0.0.1` | Type: str | **√** | **√** |
+| `--port` | `30000` | Type: int | **√** | **√** |
+| `--skip-server-warmup` | `False` | bool flag
(set to enable) | **√** | **√** |
+| `--warmups` | `None` | Type: str | **√** | **√** |
+| `--nccl-port` | `None` | Type: int | **√** | **√** |
+| `--fastapi-root-path` | `None` | Type: str | **×** | **×** |
+| `--grpc-mode` | `False` | bool flag
(set to enable) | **×** | **×** |
+| `--checkpoint-engine-`
`wait-weights-`
`before-ready` | `False` | bool flag
(set to enable) | **×** | **×** |
## Quantization and data type
@@ -45,13 +45,13 @@ click [Service Arguments](https://docs.sglang.io/advanced_features/server_argume
| `--quantization` | `None` | `modelslim` | **√** | **√** |
| `--quantization-param-path` | `None` | Type: str | **×** | **×** |
| `--kv-cache-dtype` | `auto` | `auto` | **√** | **√** |
-| `--enable-fp32-lm-head` | FALSE | bool flag
(set to enable) | **×** | **×** |
-| `--modelopt-quant` | | | **×** | **×** |
-| `--modelopt-checkpoint-`
`restore-path` | | | **×** | **×** |
-| `--modelopt-checkpoint-`
`save-path` | | | **×** | **×** |
-| `--modelopt-export-path` | | | **×** | **×** |
-| `--quantize-and-serve` | | | **×** | **×** |
-| `--rl-quant-profile` | | | **×** | **×** |
+| `--enable-fp32-lm-head` | `False` | bool flag
(set to enable) | **×** | **×** |
+| `--modelopt-quant` | `None` | Type: str | **×** | **×** |
+| `--modelopt-checkpoint-`
`restore-path` | `None` | Type: str | **×** | **×** |
+| `--modelopt-checkpoint-`
`save-path` | `None` | Type: str | **×** | **×** |
+| `--modelopt-export-path` | `None` | Type: str | **×** | **×** |
+| `--quantize-and-serve` | `False` | bool flag
(set to enable) | **×** | **×** |
+| `--rl-quant-profile` | `None` | Type: str | **×** | **×** |
## Memory and scheduling
@@ -74,7 +74,7 @@ click [Service Arguments](https://docs.sglang.io/advanced_features/server_argume
| `--swa-full-tokens-ratio` | `0.8` | Type: float | **×** | **×** |
| `--disable-hybrid-swa-memory` | `False` | bool flag
(set to enable) | **×** | **×** |
| `--abort-on-priority-`
`when-disabled` | `False` | bool flag
(set to enable) | **×** | **×** |
-| `--enable-dynamic-chunking` | | | **×** | **×** |
+| `--enable-dynamic-chunking` | `False` | bool flag
(set to enable) | **×** | **×** |
## Runtime options
@@ -96,7 +96,7 @@ click [Service Arguments](https://docs.sglang.io/advanced_features/server_argume
| `--base-gpu-id` | `0` | Type: int | **√** | **√** |
| `--gpu-id-step` | `1` | Type: int | **√** | **√** |
| `--sleep-on-idle` | `False` | bool flag (set to enable) | **√** | **√** |
-| `--custom-sigquit-handler` | `{}` | | **×** | **×** |
+| `--custom-sigquit-handler` | `None` | Optional[Callable] | **×** | **×** |
## Logging
@@ -128,25 +128,25 @@ click [Service Arguments](https://docs.sglang.io/advanced_features/server_argume
## RequestMetricsExporter configuration
-| Argument | Defaults | Options | A2 | A3 |
-|---------------------------------------|----------|---------|:--------------------------------------:|:--------------------------------------:|
-| `--export-metrics-to-`
`file` | FALSE | | **×** | **×** |
-| `--export-metrics-to-`
`file-dir` | | | **×** | **×** |
+| Argument | Defaults | Options | A2 | A3 |
+|---------------------------------------|----------|--------------------------------|:--------------------------------------:|:--------------------------------------:|
+| `--export-metrics-to-`
`file` | `False` | bool flag
(set to enable) | **×** | **×** |
+| `--export-metrics-to-`
`file-dir` | `None` | Type: str | **×** | **×** |
## API related
-| Argument | Defaults | Options | A2 | A3 |
-|-------------------------|-----------|---------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------:|:----------------------------------------:|
-| `--api-key` | `None` | Type: str | **√** | **√** |
-| `--served-model-name` | `None` | Type: str | **√** | **√** |
-| `--weight-version` | `default` | Type: str | **√** | **√** |
-| `--chat-template` | `None` | Type: str | **√** | **√** |
-| `--completion-template` | `None` | Type: str | **√** | **√** |
-| `--enable-cache-report` | `True` | bool flag
(set to enable) | **√** | **√** |
-| `--reasoning-parser` | `None` | `deepseek-r1`,
`deepseek-v3`,
`glm45`,
`gpt-oss`,
`kimi`,
`qwen3`,
`qwen3-thinking`,
`step3` | **√** | **√** |
-| `--tool-call-parser` | `None` | `llama3`,`qwen` | **√** | **√** |
-| `--sampling-defaults` | `model` | `openai`, `model` | **×** | **×** |
-| `--tool-server` | `None` | Type: str | **×** | **×** |
+| Argument | Defaults | Options | A2 | A3 |
+|-------------------------|-----------|--------------------------------|:----------------------------------------:|:----------------------------------------:|
+| `--api-key` | `None` | Type: str | **√** | **√** |
+| `--served-model-name` | `None` | Type: str | **√** | **√** |
+| `--weight-version` | `default` | Type: str | **√** | **√** |
+| `--chat-template` | `None` | Type: str | **√** | **√** |
+| `--completion-template` | `None` | Type: str | **√** | **√** |
+| `--enable-cache-report` | `False` | bool flag
(set to enable) | **√** | **√** |
+| `--reasoning-parser` | `None` | `deepseek-r1` | **√** | **√** |
+| `--tool-call-parser` | `None` | `llama`,`pythonic` | **√** | **√** |
+| `--sampling-defaults` | `model` | `openai`, `model` | **×** | **×** |
+| `--tool-server` | `None` | Type: str | **×** | **×** |
## Data parallelism
@@ -173,17 +173,17 @@ click [Service Arguments](https://docs.sglang.io/advanced_features/server_argume
## LoRA
-| Argument | Defaults | Options | A2 | A3 |
-|--------------------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------:|:----------------------------------------:|
-| `--enable-lora` | `False` | Bool flag
(set to enable) | **√** | **√** |
-| `--max-lora-rank` | `None` | Type: int | **√** | **√** |
-| `--lora-target-modules` | `None` | `q_proj`,
`k_proj`,
`v_proj`,
`o_proj`,
`gate_proj`,
`up_proj`,
`down_proj`,
`qkv_proj`,
`gate_up_proj`,
`all` | **√** | **√** |
-| `--lora-paths` | `None` | Type: List[str] /
JSON objects | **√** | **√** |
-| `--max-loras-per-batch` | `8` | Type: int | **√** | **√** |
-| `--max-loaded-loras` | `None` | Type: int | **√** | **√** |
-| `--lora-eviction-policy` | `lru` | `lru`,
`fifo` | **√** | **√** |
-| `--lora-backend` | `triton` | `triton`,
`csgmv` | **√** | **√** |
-| `--max-lora-chunk-size` | `16` | `16`, `32`,
`64`, `128` | **×** | **×** |
+| Argument | Defaults | Options | A2 | A3 |
+|--------------------------|----------|-------------------------------------|:----------------------------------------:|:----------------------------------------:|
+| `--enable-lora` | `False` | Bool flag
(set to enable) | **√** | **√** |
+| `--max-lora-rank` | `None` | Type: int | **√** | **√** |
+| `--lora-target-modules` | `None` | `all` | **√** | **√** |
+| `--lora-paths` | `None` | Type: List[str] /
JSON objects | **√** | **√** |
+| `--max-loras-per-batch` | `8` | Type: int | **√** | **√** |
+| `--max-loaded-loras` | `None` | Type: int | **√** | **√** |
+| `--lora-eviction-policy` | `lru` | `lru`,
`fifo` | **√** | **√** |
+| `--lora-backend` | `triton` | `triton` | **√** | **√** |
+| `--max-lora-chunk-size` | `16` | `16`, `32`,
`64`, `128` | **×** | **×** |
## Kernel Backends (Attention, Sampling, Grammar, GEMM)
@@ -196,29 +196,29 @@ click [Service Arguments](https://docs.sglang.io/advanced_features/server_argume
| `--grammar-backend` | `None` | `xgrammar` | **√** | **√** |
| `--mm-attention-backend` | `None` | `ascend_attn` | **√** | **√** |
| `--nsa-prefill-backend` | `flashmla_sparse` | `flashmla_sparse`,
`flashmla_decode`,
`fa3`,
`tilelang`,
`aiter` | **×** | **×** |
-| `--nsa-decode-backend` | `flashmla_kv` | `flashmla_prefill`,
`flashmla_kv`,
`fa3`,
`tilelang`,
`aiter` | **×** | **×** |
+| `--nsa-decode-backend` | `fa3` | `flashmla_prefill`,
`flashmla_kv`,
`fa3`,
`tilelang`,
`aiter` | **×** | **×** |
| `--fp8-gemm-backend` | `auto` | `auto`,
`deep_gemm`,
`flashinfer_trtllm`,
`cutlass`,
`triton`,
`aiter` | **×** | **×** |
| `--disable-flashinfer-`
`autotune` | `False` | bool flag
(set to enable) | **×** | **×** |
## Speculative decoding
-| Argument | Defaults | Options | A2 | A3 |
-|------------------------------------------------------------------|-----------|-----------------------------------------------------------------------|:----------------------------------------:|:----------------------------------------:|
-| `--speculative-algorithm` | `None` | `EAGLE`,
`EAGLE3`,
`NEXTN`,
`STANDALONE`,
`NGRAM` | **√** | **√** |
-| `--speculative-draft-model-path`
`--speculative-draft-model` | `None` | Type: str | **√** | **√** |
-| `--speculative-draft-model-`
`revision` | `None` | Type: str | **×** | **×** |
-| `--speculative-draft-load-format` | | | **×** | **×** |
-| `--speculative-num-steps` | `None` | Type: int | **√** | **√** |
-| `--speculative-eagle-topk` | `None` | Type: int | **√** | **√** |
-| `--speculative-num-draft-tokens` | `None` | Type: int | **√** | **√** |
-| `--speculative-accept-`
`threshold-single` | `1.0` | Type: float | **√** | **√** |
-| `--speculative-accept-`
`threshold-acc` | `1.0` | Type: float | **√** | **√** |
-| `--speculative-token-map` | `None` | Type: str | **×** | **×** |
-| `--speculative-attention-`
`mode` | `prefill` | `prefill`,
`decode` | **√** | **√** |
-| `--speculative-moe-runner-`
`backend` | `None` | | **√** | **√** |
-| `--speculative-moe-a2a-`
`backend` | `None` | | **√** | **√** |
-| `--speculative-draft-attention-backend` | | | **√** | **√** |
-| `--speculative-draft-model-quantization` | | | **√** | **√** |
+| Argument | Defaults | Options | A2 | A3 |
+|------------------------------------------------------------------|-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------:|:----------------------------------------:|
+| `--speculative-algorithm` | `None` | `EAGLE3`,
`NEXTN` | **√** | **√** |
+| `--speculative-draft-model-path`
`--speculative-draft-model` | `None` | Type: str | **√** | **√** |
+| `--speculative-draft-model-`
`revision` | `None` | Type: str | **√** | **√** |
+| `--speculative-draft-load-format` | `None` | `auto`,
`pt`,
`safetensors`,
`npcache`,
`dummy`,
`sharded_state`,
`gguf`,
`bitsandbytes`,
`layered`,
`flash_rl`,
`remote`,
`remote_instance`,
`fastsafetensors`,
`private` | **×** | **×** |
+| `--speculative-num-steps` | `None` | Type: int | **√** | **√** |
+| `--speculative-eagle-topk` | `None` | Type: int | **√** | **√** |
+| `--speculative-num-draft-tokens` | `None` | Type: int | **√** | **√** |
+| `--speculative-accept-`
`threshold-single` | `1.0` | Type: float | **√** | **√** |
+| `--speculative-accept-`
`threshold-acc` | `1.0` | Type: float | **√** | **√** |
+| `--speculative-token-map` | `None` | Type: str | **×** | **×** |
+| `--speculative-attention-`
`mode` | `prefill` | `prefill`,
`decode` | **√** | **√** |
+| `--speculative-moe-runner-`
`backend` | `None` | `auto`,
`deep_gemm`,
`triton`,
`triton_kernel`,
`flashinfer_trtllm`,
`flashinfer_cutlass`,
`flashinfer_mxfp4`,
`flashinfer_cutedsl`,
`cutlass` | **√** | **√** |
+| `--speculative-moe-a2a-`
`backend` | `None` | `none`,
`deepep`,
`mooncake`,
`ascend_fuseep` | **√** | **√** |
+| `--speculative-draft-attention-backend` | `None` | Type: str | **√** | **√** |
+| `--speculative-draft-model-quantization` | `None` | `awq`,
`fp8`,
`gptq`,
`marlin`,
`gptq_marlin`,
`awq_marlin`,
`bitsandbytes`,
`gguf`,
`modelopt`,
`modelopt_fp8`,
`modelopt_fp4`,
`petit_nvfp4`,
`w8a8_int8`,
`w8a8_fp8`,
`moe_wna16`,
`qoq`,
`w4afp8`,
`mxfp4`,
`auto-round`,
`compressed-tensors`,
`modelslim`,
`unquant` | **√** | **√** |
## Ngram speculative decoding
@@ -236,12 +236,12 @@ click [Service Arguments](https://docs.sglang.io/advanced_features/server_argume
| Argument | Defaults | Options | A2 | A3 |
|-------------------------------------------------------|-----------|---------------------------------------------|:----------------------------------------:|:----------------------------------------:|
-| `--expert-parallel-size`
`--ep-size`
`--ep` | `tp-size` | Type: int | **×** | **√** |
-| `--moe-a2a-backend` | `none` | `none`,
`deepep`,
`ascend_fuseep` | **×** | **√** |
-| `--moe-runner-backend` | `auto` | `auto` | **×** | **√** |
+| `--expert-parallel-size`
`--ep-size`
`--ep` | `1` | Type: int | **√** | **√** |
+| `--moe-a2a-backend` | `none` | `none`,
`deepep`,
`ascend_fuseep` | **√** | **√** |
+| `--moe-runner-backend` | `auto` | `auto`, `triton` | **√** | **√** |
| `--flashinfer-mxfp4-`
`moe-precision` | `default` | `default`,
`bf16` | **×** | **×** |
| `--enable-flashinfer-`
`allreduce-fusion` | `False` | bool flag
(set to enable) | **×** | **×** |
-| `--deepep-mode` | `auto` | `normal`,
`low_latency`,
`auto` | **×** | **√** |
+| `--deepep-mode` | `auto` | `normal`,
`low_latency`,
`auto` | **√** | **√** |
| `--deepep-config` | `None` | Type: str | **×** | **×** |
| `--ep-num-redundant-experts` | `0` | Type: int | **×** | **×** |
| `--ep-dispatch-algorithm` | `None` | Type: str | **×** | **×** |
@@ -254,8 +254,8 @@ click [Service Arguments](https://docs.sglang.io/advanced_features/server_argume
| `--expert-distribution-`
`recorder-buffer-size` | `None` | Type: int | **×** | **×** |
| `--enable-expert-distribution-`
`metrics` | `False` | bool flag
(set to enable) | **×** | **×** |
| `--moe-dense-tp-size` | `None` | Type: int | **√** | **√** |
-| `--elastic-ep-backend` | None | N/A | **×** | **×** |
-| `--mooncake-ib-device` | None | N/A | **×** | **×** |
+| `--elastic-ep-backend` | `None` | `none`, `mooncake` | **×** | **×** |
+| `--mooncake-ib-device` | `None` | Type: str | **×** | **×** |
## Mamba Cache
@@ -290,14 +290,14 @@ click [Service Arguments](https://docs.sglang.io/advanced_features/server_argume
## Ktransformer server args
-| Argument | Defaults | Options | A2 | A3 |
-|----------------------------------------------|----------|---------|:--------------------------------------:|:--------------------------------------:|
-| `--kt-weight-path` | | | **×** | **×** |
-| `--kt-method` | | | **×** | **×** |
-| `--kt-cpuinfer` | | | **×** | **×** |
-| `--kt-threadpool-count` | | | **×** | **×** |
-| `--kt-num-gpu-experts` | | | **×** | **×** |
-| `--kt-max-deferred-`
`experts-per-token` | | | **×** | **×** |
+| Argument | Defaults | Options | A2 | A3 |
+|----------------------------------------------|-----------|-----------|:--------------------------------------:|:--------------------------------------:|
+| `--kt-weight-path` | `None` | Type: str | **×** | **×** |
+| `--kt-method` | `AMXINT4` | Type: str | **×** | **×** |
+| `--kt-cpuinfer` | `None` | Type: int | **×** | **×** |
+| `--kt-threadpool-count` | 2 | Type: int | **×** | **×** |
+| `--kt-num-gpu-experts` | `None` | Type: int | **×** | **×** |
+| `--kt-max-deferred-`
`experts-per-token` | `None` | Type: int | **×** | **×** |
## Double Sparsity
@@ -331,11 +331,11 @@ click [Service Arguments](https://docs.sglang.io/advanced_features/server_argume
| Argument | Defaults | Options | A2 | A3 |
|---------------------------------------------------------|----------|--------------------------------|:----------------------------------------:|:----------------------------------------:|
| `--disable-radix-cache` | `False` | bool flag
(set to enable) | **√** | **√** |
-| `--cuda-graph-max-bs` | `None` | Type: int | **×** | **√** |
-| `--cuda-graph-bs` | `None` | List[int] | **×** | **√** |
+| `--cuda-graph-max-bs` | `None` | Type: int | **√** | **√** |
+| `--cuda-graph-bs` | `None` | List[int] | **√** | **√** |
| `--disable-cuda-graph` | `False` | bool flag
(set to enable) | **√** | **√** |
-| `--disable-cuda-graph-`
`padding` | `False` | bool flag
(set to enable) | **×** | **√** |
-| `--enable-profile-`
`cuda-graph` | `False` | bool flag
(set to enable) | **×** | **×** |
+| `--disable-cuda-graph-`
`padding` | `False` | bool flag
(set to enable) | **√** | **√** |
+| `--enable-profile-`
`cuda-graph` | `False` | bool flag
(set to enable) | **√** | **√** |
| `--enable-cudagraph-gc` | `False` | bool flag
(set to enable) | **×** | **×** |
| `--enable-nccl-nvls` | `False` | bool flag
(set to enable) | **×** | **×** |
| `--enable-symm-mem` | `False` | bool flag
(set to enable) | **×** | **×** |
@@ -343,7 +343,7 @@ click [Service Arguments](https://docs.sglang.io/advanced_features/server_argume
| `--enable-tokenizer-`
`batch-encode` | `False` | bool flag
(set to enable) | **√** | **√** |
| `--disable-tokenizer-`
`batch-encode` | `False` | bool flag
(set to enable) | **×** | **×** |
| `--disable-outlines-`
`disk-cache` | `False` | bool flag
(set to enable) | **√** | **√** |
-| `--disable-custom-`
`all-reduce` | `False` | bool flag
(set to enable) | **×** | **×** |
+| `--disable-custom-`
`all-reduce` | `False` | bool flag
(set to enable) | **√** | **√** |
| `--enable-mscclpp` | `False` | bool flag
(set to enable) | **×** | **×** |
| `--enable-torch-`
`symm-mem` | `False` | bool flag
(set to enable) | **×** | **×** |
| `--disable-overlap`
`-schedule` | `False` | bool flag
(set to enable) | **√** | **√** |
@@ -354,12 +354,12 @@ click [Service Arguments](https://docs.sglang.io/advanced_features/server_argume
| `--enable-single-`
`batch-overlap` | `False` | bool flag
(set to enable) | **×** | **×** |
| `--tbo-token-`
`distribution-threshold` | `0.48` | Type: float | **×** | **×** |
| `--enable-torch-`
`compile` | `False` | bool flag
(set to enable) | **√** | **√** |
-| `--enable-torch-`
`compile-debug-mode` | `False` | bool flag
(set to enable) | **√** | **√** |
+| `--enable-torch-`
`compile-debug-mode` | `False` | bool flag
(set to enable) | **×** | **×** |
| `--enable-piecewise-`
`cuda-graph` | `False` | bool flag
(set to enable) | **×** | **×** |
| `--piecewise-cuda-`
`graph-tokens` | `None` | Type: JSON
list | **×** | **×** |
| `--piecewise-cuda-`
`graph-compiler` | `eager` | ["eager", "inductor"] | **×** | **×** |
| `--torch-compile-max-bs` | `32` | Type: int | **×** | **×** |
-| `--piecewise-cuda-`
`graph-max-tokens` | `4096` | Type: int | **×** | **√** |
+| `--piecewise-cuda-`
`graph-max-tokens` | `4096` | Type: int | **×** | **×** |
| `--torchao-config` | `` | Type: str | **×** | **×** |
| `--enable-nan-detection` | `False` | bool flag
(set to enable) | **×** | **×** |
| `--enable-p2p-check` | `False` | bool flag
(set to enable) | **×** | **×** |
@@ -371,7 +371,7 @@ click [Service Arguments](https://docs.sglang.io/advanced_features/server_argume
| `--enable-memory-saver` | `False` | bool flag
(set to enable) | **×** | **×** |
| `--enable-weights-`
`cpu-backup` | `False` | bool flag
(set to enable) | **×** | **×** |
| `--enable-draft-weights-`
`cpu-backup` | `False` | bool flag
(set to enable) | **×** | **×** |
-| `--allow-auto-truncate` | `False` | bool flag
(set to enable) | **×** | **×** |
+| `--allow-auto-truncate` | `False` | bool flag
(set to enable) | **√** | **√** |
| `--enable-custom-`
`logit-processor` | `False` | bool flag
(set to enable) | **×** | **×** |
| `--flashinfer-mla-`
`disable-ragged` | `False` | bool flag
(set to enable) | **×** | **×** |
| `--disable-shared-`
`experts-fusion` | `False` | bool flag
(set to enable) | **×** | **×** |
@@ -386,7 +386,7 @@ click [Service Arguments](https://docs.sglang.io/advanced_features/server_argume
| `--enable-layerwise-`
`nvtx-marker` | `False` | bool flag
(set to enable) | **×** | **×** |
| `--enable-attn-tp-`
`input-scattered` | `False` | bool flag
(set to enable) | **×** | **×** |
| `--enable-nsa-prefill-`
`context-parallel` | `False` | bool flag
(set to enable) | **×** | **×** |
-| `--enable-fused-qk-`
`norm-rope` | | | **×** | **×** |
+| `--enable-fused-qk-`
`norm-rope` | `False` | bool flag
(set to enable) | **×** | **×** |
## Dynamic batch tokenizer
@@ -398,37 +398,37 @@ click [Service Arguments](https://docs.sglang.io/advanced_features/server_argume
## Debug tensor dumps
-| Argument | Defaults | Options | A2 | A3 |
-|--------------------------------------------|----------|-----------|:--------------------------------------:|:--------------------------------------:|
-| `--debug-tensor-dump-`
`output-folder` | `None` | Type: str | **×** | **×** |
-| `--debug-tensor-dump-`
`layers` | | | **×** | **×** |
-| `--debug-tensor-dump-`
`input-file` | `None` | Type: str | **×** | **×** |
-| `--debug-tensor-dump-`
`inject` | `False` | Type: str | **×** | **×** |
+| Argument | Defaults | Options | A2 | A3 |
+|--------------------------------------------|----------|-----------|:----------------------------------------:|:----------------------------------------:|
+| `--debug-tensor-dump-`
`output-folder` | `None` | Type: str | **×** | **×** |
+| `--debug-tensor-dump-`
`layers` | `None` | List[int] | **×** | **×** |
+| `--debug-tensor-dump-`
`input-file` | `None` | Type: str | **√** | **√** |
+| `--debug-tensor-dump-`
`inject` | `False` | Type: str | **×** | **×** |
## PD disaggregation
-| Argument | Defaults | Options | A2 | A3 |
-|---------------------------------------------------------|------------|-----------------------------------------------------|:----------------------------------------:|:----------------------------------------:|
-| `--disaggregation-mode` | `null` | `null`,
`prefill`,
`decode` | **√** | **√** |
-| `--disaggregation-transfer-backend` | `mooncake` | `mooncake`,
`nixl`,
`ascend`,
`fake` | **√** | **√** |
-| `--disaggregation-bootstrap-port` | `8998` | Type: int | **√** | **√** |
-| `--disaggregation-decode-tp` | `None` | Type: int | **×** | **×** |
-| `--disaggregation-decode-dp` | `None` | Type: int | **×** | **×** |
-| `--disaggregation-prefill-pp` | `1` | Type: int | **×** | **×** |
-| `--disaggregation-ib-device` | `None` | Type: str | **×** | **×** |
-| `--disaggregation-decode-`
`enable-offload-kvcache` | `False` | bool flag
(set to enable) | **×** | **×** |
-| `--disaggregation-decode-`
`enable-fake-auto` | `False` | bool flag
(set to enable) | **×** | **×** |
-| `--num-reserved-decode-tokens` | `512` | Type: int | **√** | **√** |
-| `--disaggregation-decode-`
`polling-interval` | `1` | Type: int | **√** | **√** |
+| Argument | Defaults | Options | A2 | A3 |
+|---------------------------------------------------------|------------|---------------------------------------|:----------------------------------------:|:----------------------------------------:|
+| `--disaggregation-mode` | `null` | `null`,
`prefill`,
`decode` | **√** | **√** |
+| `--disaggregation-transfer-backend` | `mooncake` | `ascend` | **√** | **√** |
+| `--disaggregation-bootstrap-port` | `8998` | Type: int | **√** | **√** |
+| `--disaggregation-decode-tp` | `None` | Type: int | **√** | **√** |
+| `--disaggregation-decode-dp` | `None` | Type: int | **√** | **√** |
+| `--disaggregation-prefill-pp` | `1` | Type: int | **×** | **×** |
+| `--disaggregation-ib-device` | `None` | Type: str | **×** | **×** |
+| `--disaggregation-decode-`
`enable-offload-kvcache` | `False` | bool flag
(set to enable) | **×** | **×** |
+| `--disaggregation-decode-`
`enable-fake-auto` | `False` | bool flag
(set to enable) | **×** | **×** |
+| `--num-reserved-decode-tokens` | `512` | Type: int | **√** | **√** |
+| `--disaggregation-decode-`
`polling-interval` | `1` | Type: int | **√** | **√** |
## Encode prefill disaggregation
-| Argument | Defaults | Options | A2 | A3 |
-|------------------------------|----------|---------|:--------------------------------------:|:--------------------------------------:|
-| `--encoder-only` | | | **×** | **×** |
-| `--language-only` | | | **×** | **×** |
-| `--encoder-transfer-backend` | | | **×** | **×** |
-| `--encoder-urls` | | | **×** | **×** |
+| Argument | Defaults | Options | A2 | A3 |
+|------------------------------|--------------------|----------------------------------------------------------------|:--------------------------------------:|:--------------------------------------:|
+| `--encoder-only` | `False` | bool flag
(set to enable) | **×** | **×** |
+| `--language-only` | `False` | bool flag
(set to enable) | **×** | **×** |
+| `--encoder-transfer-backend` | `zmq_to_scheduler` | `zmq_to_scheduler`,
`zmq_to_tokenizer`,
`mooncake` | **×** | **×** |
+| `--encoder-urls` | `[]` | List[str] | **×** | **×** |
## Custom weight loader
@@ -439,8 +439,8 @@ click [Service Arguments](https://docs.sglang.io/advanced_features/server_argume
| `--remote-instance-weight-`
`loader-seed-instance-ip` | `None` | Type: str | **×** | **×** |
| `--remote-instance-weight-`
`loader-seed-instance-service-port` | `None` | Type: int | **×** | **×** |
| `--remote-instance-weight-`
`loader-send-weights-group-ports` | `None` | Type: JSON
list | **×** | **×** |
-| `--remote-instance-weight-`
`loader-backend` | | | **×** | **×** |
-| `--remote-instance-weight-`
`loader-start-seed-via-transfer-engine` | | | **×** | **×** |
+| `--remote-instance-weight-`
`loader-backend` | `nccl` | Type: str | **×** | **×** |
+| `--remote-instance-weight-`
`loader-start-seed-via-transfer-engine` | `False` | bool flag
(set to enable) | **×** | **×** |
## For PD-Multiplexing
@@ -452,22 +452,22 @@ click [Service Arguments](https://docs.sglang.io/advanced_features/server_argume
## For Multi-Modal
-| Argument | Defaults | Options | A2 | A3 |
-|-----------------------------------------------|----------|-------------------|------------------------------------------|------------------------------------------|
-| `--mm-max-concurrent-calls` | | | **×** | **×** |
-| `--mm-per-request-timeout` | | | **×** | **×** |
-| `--enable-broadcast-mm-`
`inputs-process` | | | **×** | **×** |
-| `--mm-process-config` | | | **×** | **×** |
-| `--mm-enable-dp-encoder` | | | **×** | **×** |
-| `--limit-mm-data-per-request` | `None` | Type: JSON / Dict | **√** | **√** |
+| Argument | Defaults | Options | A2 | A3 |
+|-----------------------------------------------|----------|--------------------------------|------------------------------------------|------------------------------------------|
+| `--mm-max-concurrent-calls` | 32 | Type: int | **×** | **×** |
+| `--mm-per-request-timeout` | 10.0 | Type: float | **×** | **×** |
+| `--enable-broadcast-mm-`
`inputs-process` | `False` | bool flag
(set to enable) | **×** | **×** |
+| `--mm-process-config` | `None` | Type: JSON / Dict | **×** | **×** |
+| `--mm-enable-dp-encoder` | `False` | bool flag
(set to enable) | **×** | **×** |
+| `--limit-mm-data-per-request` | `None` | Type: JSON / Dict | **√** | **√** |
## For checkpoint decryption
-| Argument | Defaults | Options | A2 | A3 |
-|---------------------------------|----------|---------|:--------------------------------------:|:--------------------------------------:|
-| `--decrypted-config-file` | | | **×** | **×** |
-| `--decrypted-draft-config-file` | | | **×** | **×** |
-| `--enable-prefix-mm-cache` | | | **×** | **×** |
+| Argument | Defaults | Options | A2 | A3 |
+|---------------------------------|----------|--------------------------------|:--------------------------------------:|:--------------------------------------:|
+| `--decrypted-config-file` | `None` | Type: str | **×** | **×** |
+| `--decrypted-draft-config-file` | `None` | Type: str | **×** | **×** |
+| `--enable-prefix-mm-cache` | `False` | bool flag
(set to enable) | **×** | **×** |
## For deterministic inference
@@ -483,6 +483,6 @@ click [Service Arguments](https://docs.sglang.io/advanced_features/server_argume
## Configuration file support
-| Argument | Defaults | Options | A2 | A3 |
-|------------|----------|---------|----------------------------------------|----------------------------------------|
-| `--config` | | yaml | **×** | **×** |
+| Argument | Defaults | Options | A2 | A3 |
+|------------|----------|-----------|----------------------------------------|----------------------------------------|
+| `--config` | `None` | Type: str | **×** | **×** |
diff --git a/docs/platforms/ascend_npu_support_models.md b/docs/platforms/ascend_npu_support_models.md
index f397edba2..3a7a3af96 100644
--- a/docs/platforms/ascend_npu_support_models.md
+++ b/docs/platforms/ascend_npu_support_models.md
@@ -1,7 +1,7 @@
# Support Models on Ascend NPU
This section describes the models supported on the Ascend NPU, including Large Language Models, Multimodal Language
-Models, Embedding Models, and Rerank Models. Mainstream DeepSeek/Qwen/GLM series are included.
+Models, Embedding Models, Reward Models and Rerank Models. Mainstream DeepSeek/Qwen/GLM series are included.
You are welcome to enable various models based on your business requirements.
## Large Language Models
@@ -40,10 +40,10 @@ You are welcome to enable various models based on your business requirements.
| inclusionAI/Ling-lite | Ling | **√** | **√** |
| ibm-granite/granite-3.1-8b-instruct | Granite | **√** | **√** |
| ibm-granite/granite-3.0-3b-a800m-instruct | Granite MoE | **√** | **√** |
-| databricks/dbrx-instruct | DBRX (Databricks) | **×** | **×** |
-| baichuan-inc/Baichuan2-13B-Chat | Baichuan 2 (7B, 13B) | **×** | **×** |
-| baidu/ERNIE-4.5-21B-A3B-PT | ERNIE-4.5 (4.5, 4.5MoE series) | **×** | **×** |
-| openbmb/MiniCPM3-4B | MiniCPM (v3, 4B) | **×** | **×** |
+| databricks/dbrx-instruct | DBRX (Databricks) | **√** | **√** |
+| baichuan-inc/Baichuan2-13B-Chat | Baichuan 2 (7B, 13B) | **√** | **√** |
+| baidu/ERNIE-4.5-21B-A3B-PT | ERNIE-4.5 (4.5, 4.5MoE series) | **√** | **√** |
+| openbmb/MiniCPM3-4B | MiniCPM (v3, 4B) | **√** | **√** |
| openai/gpt-oss-120b | GPTOSS | **×** | **×** |
## Multimodal Language Models