[diffusion] quant: add support for svdquant and nunchaku (#18549)

Co-authored-by: AichenF <aichenf@nvidia.com> Co-authored-by: jianyingzhu <53300651@qq.com>
2026-02-15 20:43:00 +08:00
parent 88010e9601
commit 3feb48139e
20 changed files with 1304 additions and 123 deletions
--- a/python/sglang/multimodal_gen/runtime/managers/gpu_worker.py
+++ b/python/sglang/multimodal_gen/runtime/managers/gpu_worker.py
@@ -352,6 +352,7 @@ OOM detected. Possible solutions:
    2. Enable SP and/or TP
    3. Opt for a sparse-attention backend
    4. Enable FSDP by `--use-fsdp-inference` (in a multi-GPU setup)
+    5. Enable quantization (e.g. nunchaku)
  Or, open an issue on GitHub https://github.com/sgl-project/sglang/issues/new/choose
 """