From 20ca2c6e1ebc8c830e9f92de7345200348ea8e14 Mon Sep 17 00:00:00 2001
From: Hexq0210 <893781835@qq.com>
Date: Thu, 8 Jan 2026 21:50:42 +0800
Subject: [PATCH] [NPU] update model and features supported (#16733)
---
docs/platforms/ascend_npu_support_features.md | 20 ++--
docs/platforms/ascend_npu_support_models.md | 91 ++++++++++---------
2 files changed, 57 insertions(+), 54 deletions(-)
diff --git a/docs/platforms/ascend_npu_support_features.md b/docs/platforms/ascend_npu_support_features.md
index 55b7360d0..0a0060d8b 100644
--- a/docs/platforms/ascend_npu_support_features.md
+++ b/docs/platforms/ascend_npu_support_features.md
@@ -276,7 +276,7 @@ click [Server Arguments](https://docs.sglang.io/advanced_features/server_argumen
| `--hicache-size` | `0` | Type: int | **√** | **√** |
| `--hicache-write-policy` | `write_through` | `write_back`,
`write_through`,
`write_through_selective` | **√** | **√** |
| `--radix-eviction-policy` | `lru` | `lru`, `lfu` | **√** | **√** |
-| `--hicache-io-backend` | `kernel` | `kernel_ascend` | **√** | **√** |
+| `--hicache-io-backend` | `kernel` | `kernel_ascend`,
`direct` | **√** | **√** |
| `--hicache-mem-layout` | `layer_first` | `page_first_direct`,
`page_first_kv_split` | **√** | **√** |
| `--hicache-storage-`
`backend` | `None` | `file` | **√** | **√** |
| `--hicache-storage-`
`prefetch-policy` | `best_effort` | `best_effort`,
`wait_complete`,
`timeout` | **×** | **×** |
@@ -432,15 +432,15 @@ click [Server Arguments](https://docs.sglang.io/advanced_features/server_argumen
## Custom weight loader
-| Argument | Defaults | Options | A2 | A3 |
-|-------------------------------------------------------------------------|----------|--------------------------------|:----------------------------------------:|------------------------------------------|
-| `--custom-weight-loader` | `None` | List[str] | **×** | **×** |
-| `--weight-loader-disable-`
`mmap` | `False` | bool flag
(set to enable) | **√** | **√** |
-| `--remote-instance-weight-`
`loader-seed-instance-ip` | `None` | Type: str | **×** | **×** |
-| `--remote-instance-weight-`
`loader-seed-instance-service-port` | `None` | Type: int | **×** | **×** |
-| `--remote-instance-weight-`
`loader-send-weights-group-ports` | `None` | Type: JSON
list | **×** | **×** |
-| `--remote-instance-weight-`
`loader-backend` | `nccl` | Type: str | **×** | **×** |
-| `--remote-instance-weight-`
`loader-start-seed-via-transfer-engine` | `False` | bool flag
(set to enable) | **×** | **×** |
+| Argument | Defaults | Options | A2 | A3 |
+|-------------------------------------------------------------------------|----------|---------------------------------|:----------------------------------------:|------------------------------------------|
+| `--custom-weight-loader` | `None` | List[str] | **×** | **×** |
+| `--weight-loader-disable-`
`mmap` | `False` | bool flag
(set to enable) | **√** | **√** |
+| `--remote-instance-weight-`
`loader-seed-instance-ip` | `None` | Type: str | **×** | **×** |
+| `--remote-instance-weight-`
`loader-seed-instance-service-port` | `None` | Type: int | **×** | **×** |
+| `--remote-instance-weight-`
`loader-send-weights-group-ports` | `None` | Type: JSON
list | **×** | **×** |
+| `--remote-instance-weight-`
`loader-backend` | `nccl` | `transfer_engine`,
`nccl` | **×** | **×** |
+| `--remote-instance-weight-`
`loader-start-seed-via-transfer-engine` | `False` | bool flag
(set to enable) | **×** | **×** |
## For PD-Multiplexing
diff --git a/docs/platforms/ascend_npu_support_models.md b/docs/platforms/ascend_npu_support_models.md
index 3a7a3af96..701aa1ac7 100644
--- a/docs/platforms/ascend_npu_support_models.md
+++ b/docs/platforms/ascend_npu_support_models.md
@@ -6,55 +6,58 @@ You are welcome to enable various models based on your business requirements.
## Large Language Models
-| Models | Model Family | A2 Supported | A3 Supported |
-|-------------------------------------------|--------------------------------|:----------------------------------------:|:----------------------------------------:|
-| DeepSeek V3/V3.1 | DeepSeek | **√** | **√** |
-| DeepSeek V3.2 | DeepSeek | **√** | **√** |
-| DeepSeek R1 | DeepSeek | **√** | **√** |
-| DeepSeek V2 | DeepSeek | **√** | **√** |
-| Qwen3 | Qwen | **√** | **√** |
-| Qwen3-MoE | Qwen | **√** | **√** |
-| Qwen3-Next | Qwen | **√** | **√** |
-| Qwen3-Coder | Qwen | **√** | **√** |
-| Qwen2.5 | Qwen | **√** | **√** |
-| QwQ-32B | Qwen | **√** | **√** |
-| meta-llama/Llama-4-Scout-17B-16E-Instruct | Llama | **√** | **√** |
-| AI-ModelScope/Llama-3.1-8B-Instruct | Llama | **√** | **√** |
-| LLM-Research/Llama-3.2-1B-Instruct | Llama | **√** | **√** |
-| mistralai/Mistral-7B-Instruct-v0.2 | Mistral | **√** | **√** |
-| google/gemma-3-4b-it | Gemma | **√** | **√** |
-| microsoft/Phi-4-multimodal-instruct | Phi | **√** | **√** |
-| allenai/OLMoE-1B-7B-0924 | OLMoE | **√** | **√** |
-| stabilityai/stablelm-2-1_6b | StableLM | **√** | **√** |
-| CohereForAI/c4ai-command-r-v01 | Command-R | **√** | **√** |
-| huihui-ai/grok-2 | Grok | **√** | **√** |
-| ZhipuAI/chatglm2-6b | ChatGLM | **√** | **√** |
-| Shanghai_AI_Laboratory/internlm2-7b | InternLM 2 | **√** | **√** |
-| LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct | ExaONE 3 | **√** | **√** |
-| xverse/XVERSE-MoE-A36B | XVERSE | **√** | **√** |
-| HuggingFaceTB/SmolLM-1.7B | SmolLM | **√** | **√** |
-| ZhipuAI/glm-4-9b-chat | GLM-4 | **√** | **√** |
-| XiaomiMiMo/MiMo-7B-RL | MiMo | **√** | **√** |
-| arcee-ai/AFM-4.5B-Base | Arcee AFM-4.5B | **√** | **√** |
-| Howeee/persimmon-8b-chat | Persimmon | **√** | **√** |
-| inclusionAI/Ling-lite | Ling | **√** | **√** |
-| ibm-granite/granite-3.1-8b-instruct | Granite | **√** | **√** |
-| ibm-granite/granite-3.0-3b-a800m-instruct | Granite MoE | **√** | **√** |
-| databricks/dbrx-instruct | DBRX (Databricks) | **√** | **√** |
-| baichuan-inc/Baichuan2-13B-Chat | Baichuan 2 (7B, 13B) | **√** | **√** |
-| baidu/ERNIE-4.5-21B-A3B-PT | ERNIE-4.5 (4.5, 4.5MoE series) | **√** | **√** |
-| openbmb/MiniCPM3-4B | MiniCPM (v3, 4B) | **√** | **√** |
-| openai/gpt-oss-120b | GPTOSS | **×** | **×** |
+| Models | Model Family | A2 Supported | A3 Supported |
+|--------------------------------------------|--------------------------------|:----------------------------------------:|:----------------------------------------:|
+| DeepSeek V3/V3.1 | DeepSeek | **√** | **√** |
+| vllm-ascend/DeepSeek-V3.2-Exp-W8A8 | DeepSeek | **√** | **√** |
+| vllm-ascend/DeepSeek-R1-0528-W8A8 | DeepSeek | **√** | **√** |
+| vllm-ascend/DeepSeek-V2-Lite-W8A8 | DeepSeek | **√** | **√** |
+| Qwen/Qwen3-30B-A3B-Instruct-2507 | Qwen | **√** | **√** |
+| Qwen/Qwen3-32B | Qwen | **√** | **√** |
+| Qwen/Qwen3-0.6B | Qwen | **√** | **√** |
+| vllm-ascend/Qwen3-235B-A22B-W8A8 | Qwen | **√** | **√** |
+| Qwen/Qwen3-Next-80B-A3B-Instruct | Qwen | **√** | **√** |
+| Qwen3-Coder-480B-A35B-Instruct-w8a8-QuaRot | Qwen | **√** | **√** |
+| Qwen/Qwen2.5-7B-Instruct | Qwen | **√** | **√** |
+| vllm-ascend/QWQ-32B-W8A8 | Qwen | **√** | **√** |
+| meta-llama/Llama-4-Scout-17B-16E-Instruct | Llama | **√** | **√** |
+| AI-ModelScope/Llama-3.1-8B-Instruct | Llama | **√** | **√** |
+| LLM-Research/Llama-3.2-1B-Instruct | Llama | **√** | **√** |
+| mistralai/Mistral-7B-Instruct-v0.2 | Mistral | **√** | **√** |
+| google/gemma-3-4b-it | Gemma | **√** | **√** |
+| microsoft/Phi-4-multimodal-instruct | Phi | **√** | **√** |
+| allenai/OLMoE-1B-7B-0924 | OLMoE | **√** | **√** |
+| stabilityai/stablelm-2-1_6b | StableLM | **√** | **√** |
+| CohereForAI/c4ai-command-r-v01 | Command-R | **√** | **√** |
+| huihui-ai/grok-2 | Grok | **√** | **√** |
+| ZhipuAI/chatglm2-6b | ChatGLM | **√** | **√** |
+| Shanghai_AI_Laboratory/internlm2-7b | InternLM 2 | **√** | **√** |
+| LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct | ExaONE 3 | **√** | **√** |
+| xverse/XVERSE-MoE-A36B | XVERSE | **√** | **√** |
+| HuggingFaceTB/SmolLM-1.7B | SmolLM | **√** | **√** |
+| ZhipuAI/glm-4-9b-chat | GLM-4 | **√** | **√** |
+| XiaomiMiMo/MiMo-7B-RL | MiMo | **√** | **√** |
+| arcee-ai/AFM-4.5B-Base | Arcee AFM-4.5B | **√** | **√** |
+| Howeee/persimmon-8b-chat | Persimmon | **√** | **√** |
+| inclusionAI/Ling-lite | Ling | **√** | **√** |
+| ibm-granite/granite-3.1-8b-instruct | Granite | **√** | **√** |
+| ibm-granite/granite-3.0-3b-a800m-instruct | Granite MoE | **√** | **√** |
+| databricks/dbrx-instruct | DBRX (Databricks) | **√** | **√** |
+| baichuan-inc/Baichuan2-13B-Chat | Baichuan 2 (7B, 13B) | **√** | **√** |
+| baidu/ERNIE-4.5-21B-A3B-PT | ERNIE-4.5 (4.5, 4.5MoE series) | **√** | **√** |
+| openbmb/MiniCPM3-4B | MiniCPM (v3, 4B) | **√** | **√** |
+| openai/gpt-oss-120b | GPTOSS | **×** | **×** |
## Multimodal Language Models
| Models | Model Family (Variants) | A2 Supported | A3 Supported |
|-----------------------------------------------|---------------------------|------------------------------------------|:----------------------------------------:|
-| Qwen2.5-VL-72B-Instruct-w8a8 | Qwen-VL | **√** | **√** |
-| Qwen3-VL-30B-A3B-Instruct | Qwen-VL | **√** | **√** |
-| Qwen3-VL-8B-Instruct | Qwen-VL | **√** | **√** |
-| Qwen3-VL-4B-Instruct | Qwen-VL | **√** | **√** |
-| Qwen3-VL-235B-A22B-Instruct | Qwen-VL | **√** | **√** |
+| Qwen/Qwen2.5-VL-3B-Instruct | Qwen-VL | **√** | **√** |
+| Qwen/Qwen2.5-VL-72B-Instruct | Qwen-VL | **√** | **√** |
+| Qwen/Qwen3-VL-30B-A3B-Instruct | Qwen-VL | **√** | **√** |
+| Qwen/Qwen3-VL-8B-Instruct | Qwen-VL | **√** | **√** |
+| Qwen/Qwen3-VL-4B-Instruct | Qwen-VL | **√** | **√** |
+| Qwen/Qwen3-VL-235B-A22B-Instruct | Qwen-VL | **√** | **√** |
| deepseek-ai/deepseek-vl2 | DeepSeek-VL2 | **√** | **√** |
| deepseek-ai/Janus-Pro-7B | Janus-Pro (1B, 7B) | **√** | **√** |
| openbmb/MiniCPM-V-2_6 | MiniCPM-V / MiniCPM-o | **√** | **√** |