chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Enable Nvidia's ModelOpt fp8 quantized models (#2535)

Browse Source
This commit is contained in:
Zhiyu
2025-01-06 14:54:52 -08:00
committed by GitHub
parent b8574f6953
commit 287427e2e6
5 changed files with 185 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
python/sglang/srt/layers/linear.py
View File

@@ -44,6 +44,7 @@ WEIGHT_LOADER_V2_SUPPORTED = [
"MarlinLinearMethod",
"GPTQLinearMethod",
"QQQLinearMethod",
"ModelOptFp8LinearMethod",
]