Support Models on Ascend NPU

This section describes the models supported on the Ascend NPU, including Large Language Models, Multimodal Language Models, Embedding Models, and Rerank Models. Mainstream DeepSeek/Qwen/GLM series are included. You are welcome to enable various models based on your business requirements.

Large Language Models

Model Family	Recommend Models	A2 Supported	A3 Supported
DeepSeek	DeepSeek V1, V2, V3(V3.1,V3.2), R1	√	√
Qwen	Qwen 3, Qwen 3Moe	√	√
Llama	meta-llama/Llama-4-Scout-17B-16E-Instruct, AI-ModelScope/Llama-3.1-8B-Instruct, LLM-Research/Llama-3.2-1B-Instruct	×	×
Mistral	mistralai/Mistral-7B-Instruct-v0.2	√	√
Gemma	google/gemma-3-4b-it	√	√
Phi	microsoft/Phi-4-multimodal-instruct	√	√
OLMoE	allenai/OLMoE-1B-7B-0924	×	×
StableLM	stabilityai/stablelm-2-1_6b	×	×
Command-R	CohereForAI/c4ai-command-r-v01	×	×
Grok	huihui-ai/grok-2	×	×
ChatGLM	ZhipuAI/chatglm2-6b	×	×
InternLM 2	Shanghai_AI_Laboratory/internlm2-7b	√	√
ExaONE 3	LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct	√	√
XVERSE	xverse/XVERSE-MoE-A36B	√	√
SmolLM	HuggingFaceTB/SmolLM-1.7B	√	√
GLM-4	ZhipuAI/glm-4-9b-chat	×	×
MiMo	XiaomiMiMo/MiMo-7B-RL	√	√
Arcee AFM-4.5B	arcee-ai/AFM-4.5B-Base	√	√
Persimmon	Howeee/persimmon-8b-chat	√	√
Ling	inclusionAI/Ling-lite	√	√
Granite	ibm-granite/granite-3.1-8b-instruct	√	√
Granite Moe	ibm-granite/granite-3.0-3b-a800m-instruct	√	√
DBRX (Databricks)	databricks/dbrx-instruct	×	×
Baichuan 2 (7B, 13B)	baichuan-inc/Baichuan2-13B-Chat	×	×
ERNIE-4.5 (4.5, 4.5MoE series)	baidu/ERNIE-4.5-21B-A3B-PT	×	×
MiniCPM (v3, 4B)	openbmb/MiniCPM3-4B	×	×
GPTOSS	openai/gpt-oss-120b	×	×

Multimodal Language Models

Model Family	Recommend Models	A2 Supported	A3 Supported
Qwen-VL (Qwen2 series)	Qwen/Qwen3-VL-235B-A22B-Instruct	×	×
DeepSeek-VL2	deepseek-ai/deepseek-vl2	×	×
Janus-Pro (1B, 7B)	deepseek-ai/Janus-Pro-7B	√	√
MiniCPM-V / MiniCPM-o	openbmb/MiniCPM-V-2_6	×	×
Gemma 3 (Multimodal)	google/gemma-3-4b-it	√	√
Mistral-Small-3.1-24B	mistralai/Mistral-Small-3.1-24B-Instruct-2503	×	×
Phi-4-multimodal-instruct	microsoft/Phi-4-multimodal-instruct	×	×
MiMo-VL (7B)	XiaomiMiMo/MiMo-VL-7B-RL	×	×
LLaVA (v1.5 & v1.6)	AI-ModelScope/llava-v1.6-34b	√	√
LLaVA-NeXT (8B, 72B)	lmms-lab/llava-next-72b	√	√
LLaVA-OneVision	lmms-lab/llava-onevision-qwen2-7b-ov	×	×
Kimi-VL (A3B)	Kimi/Kimi-VL-A3B-Instruct	×	×
GLM-4.5V (106B) / GLM-4.1V(9B)	ZhipuAI/GLM-4.5V	×	√
Llama 3.2 Vision (11B)	meta-llama/Llama-3.2-11B-Vision-Instruct	×	×

Embedding Models

Model Family	Recommend Models	A2 Supported	A3 Supported
E5 (Llama/Mistral based)	intfloat/e5-mistral-7b-instruct	×	×
GTE-Qwen2	iic/gte_Qwen2-1.5B-instruct	×	×
Qwen3-Embedding	Qwen/Qwen3-Embedding-8B	×	×
GME (Multimodal)	Alibaba-NLP/gme-Qwen2-VL-2B-Instruct	×	×
CLIP	AI-ModelScope/clip-vit-large-patch14-336	×	√
BGE	BAAI/bge-large-en-v1.5	×	×

Reward Models

Model Family	Recommend Models	A2 Supported	A3 Supported
Llama3.1 Reward	Skywork/Skywork-Reward-Llama-3.1-8B-v0.2	×	√
InternLM 2 Reward	Shanghai_AI_Laboratory/internlm2-7b-reward	×	√
Qwen2.5 Reward - Math	Qwen/Qwen2.5-Math-RM-72B	×	√
Qwen2.5 Reward - Sequence	jason9693/Qwen2.5-1.5B-apeach	×	√
Gemma 2-27B Reward	Skywork/Skywork-Reward-Gemma-2-27B-v0.2	×	×

Rerank Models

Model Family	Recommend Models	A2 Supported	A3 Supported
BGE-Reranker	BAAI/bge-reranker-v2-m3	×	×

13 KiB Raw Blame History Unescape Escape