[NPU] NPU quantization refactoring & more quantization formats support (#14504)

Co-authored-by: TamirBaydasov <mr.jeijy@gmail.com>
Co-authored-by: Tamir Baydasov <41994229+TamirBaydasov@users.noreply.github.com>
Co-authored-by: Савкин Артем <savkinartem@MacBook-Air-Viktoria.local>
Co-authored-by: Edward Shogulin <edward.shogulin@gmail.com>
This commit is contained in:
Артем Савкин
2026-01-15 04:25:15 +08:00
committed by GitHub
co-authored by TamirBaydasov Tamir Baydasov Савкин Артем Edward Shogulin
parent f091858304
commit 424a380077
30 changed files with 1962 additions and 771 deletions
@@ -30,7 +30,6 @@ python3 -m sglang.launch_server \
--trust-remote-code \
--attention-backend ascend \
--device npu \
--quantization modelslim \
--watchdog-timeout 9000 \
--cuda-graph-bs 8 16 24 28 32 \
--mem-fraction-static 0.68 \
@@ -88,7 +87,6 @@ python -m sglang.launch_server \
--mem-fraction-static 0.6 \
--attention-backend ascend \
--device npu \
--quantization modelslim \
--max-running-requests 8 \
--context-length 8192 \
--disable-radix-cache \
@@ -144,7 +142,6 @@ python -m sglang.launch_server \
--max-running-requests 352 \
--attention-backend ascend \
--device npu \
--quantization modelslim \
--moe-a2a-backend deepep \
--enable-dp-attention \
--deepep-mode low_latency \
@@ -216,7 +213,6 @@ do
--mem-fraction-static 0.81 \
--attention-backend ascend \
--device npu \
--quantization modelslim \
--max-running-requests 8 \
--context-length 8192 \
--disable-radix-cache \
@@ -279,7 +275,6 @@ do
--max-running-requests 832 \
--attention-backend ascend \
--device npu \
--quantization modelslim \
--moe-a2a-backend deepep \
--enable-dp-attention \
--deepep-mode low_latency \