chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
8,573 Commits 1 Branch 0 Tags
9a9f996f8de7bc51a007ad3d79dc4b0a03b9a9d4
Commit Graph

83 Commits

Author SHA1 Message Date
Binyao Jiang
9a9f996f8d [FP8] Fix weight_scale shape to match with x_scale shape for per-tensor quant under torch.compile (#16356) 2026-01-05 13:55:26 -08:00
haoyangli-amd
9327482baa [bugfix][quark] Fixed an issue where per_token could not be properly recognized when the token count was 1. (#14415) 2025-12-15 22:54:31 -08:00
b8zhong
c51efb8b84 fix fp8 gemm nightly CI (#14844) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-10 15:57:51 -08:00
b8zhong
55504df2f7 Add FP8 Blockwise GEMM Backend Flag --fp8-gemm-backend (#14379) 2025-12-09 12:05:56 -08:00
Daniel Cámpora
8428078436 Add Mistral Large 3 support. (#14213) 
Co-authored-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Linda-Stadter <57756729+Linda-Stadter@users.noreply.github.com>
 2025-12-04 20:00:05 +08:00
fzyzcjy
f87b8eab23 Tiny fix transform_scale_ue8m0 wrong output in some scenarios (#14003) 2025-12-01 14:45:27 +08:00
qichu-yun
bd0e690857 [Feature] Enable PTPC FP8 for compressed tensors moe (aiter kernel) (#12181) 2025-11-30 21:54:28 -08:00
Kangyan-Zhou
1d3d8b3418 Fix Minimax M2 loading issue (#13956) 2025-11-29 17:07:19 -05:00
fzyzcjy
2bc8ee8b74 Tiny support 3D tensors in inverse_transform_scale_ue8m0 (#14002) 2025-11-27 20:20:45 +08:00
yctseng0211
c8ede0e93c [ROCM] Optimized deepseek-r1 fp8 model with + triton_gemm_a8w8 + batch_gemm_a8w8 + fused set_mla_kv_buffer kernel (#13617) 
Co-authored-by: root <root@smci355-ccs-aus-m12-17.cs-aus.dcgpu>
Co-authored-by: jacky.cheng <yichiche@amd.com>
 2025-11-20 00:29:56 -08:00
Kaixi Hou
c3c4da71fb [NVIDIA] Add fp8 gemm benchmark on blackwell (#13528) 2025-11-19 19:35:00 -08:00
b8zhong
f33860777c [Piecewise CUDA Graph] Support ModelOpt FP8 (#13094) 2025-11-18 12:46:24 +08:00
fzyzcjy
d971f22898 Super tiny expose transform_scale_ue8m0 API for RL frameworks (#13323) 2025-11-15 17:31:04 +08:00
fzyzcjy
8e6083bfcf Support inverse transform ue8m0 scale (#13285) 2025-11-15 16:34:32 +08:00
Kaixi Hou
5ae0ac4244 [NVIDIA] Fix use case of SGLANG_ENABLE_FLASHINFER_GEMM (#13274) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-11-14 12:51:11 -08:00
Bowen Bao
67e9d287ee [Quantization] Support Quark Dense + MoE FP8 & FP8 PTPC (#10485) 
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com>
 2025-11-13 08:16:00 -08:00
yctseng0211
4a78031a71 [ROCM] Optimized deepseek-r1 model with rmsnorm + fp8 quant fusion (#12689) 
should be clean after https://github.com/sgl-project/sglang/pull/13017 landed
 2025-11-11 02:59:10 -08:00
Liangsheng Yin
665416f6dd Unify memory management across (overlap, non-overlap) x (page>=1) x (spec, non-spec, spec v2) x (retract, finished) (#12224) 2025-11-11 02:56:22 +08:00
weiliang
88596739a4 Support running FP4 Deepseek on SM120. (#11708) 2025-10-27 17:37:49 -07:00
fzyzcjy
12eb02e982 Change bf16 to fp8 for some gemms in attention for DeepSeek ckpt v2 (#11805) 2025-10-19 16:15:13 +08:00
Cheng Wan
5b214b50b6 [Refactor] move deep_gemm_wrapper out of quantization (#11784) 2025-10-17 18:57:54 -07:00
Lianmin Zheng
9b8ebb2798 move more files under srt/utils (#11285) 2025-10-09 16:46:15 -07:00
sogalin
c32fb7a24d [ROCm] Fix fp8 quantization accuracy issue. (#10558) 2025-09-17 17:44:59 -07:00
fzyzcjy
fa46e2bd40 Support offloading in fp8 (#9948) 2025-09-14 01:14:28 -07:00
Morpheus Guo
4efe844a25 enable aiter gemm_a8w8_bpreshuffle for ptpc gemm (#8555) 2025-09-05 12:54:40 -07:00
fzyzcjy
5e5c30d9ab Tiny let DeepGEMM scale checks cover more cases (#7182) 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-09-05 19:52:32 +08:00
PGFLMG
aa3eba8eb4 [sgl-kernel] misc: update deepgemm version for sgl-kernel (#9340) 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-08-27 12:01:30 -07:00
Lianmin Zheng
fd71b11b1d move is_sm90_supported/is_sm100_supported to python/sglang/srt/utils.py (#9679) 2025-08-27 03:34:29 -07:00
Hongbo Xu
9c8e4f69c3 [5/n]decouple quantization implementation from vLLM dependency (#9454) 2025-08-21 12:52:07 -07:00
Martin Vit
7cd2ee06d7 feat: Add Triton fallback option and SM120 MoE configs for FP8 models (#9251) 2025-08-20 19:33:15 -07:00
Enrique Shockwave
e483ab6d20 enable marlin fp8 blockwise (#8990) 2025-08-18 18:53:15 -07:00
Cheng Wan
295895120d [6/N] MoE Refactor: Cleanup MoE-related configs (#8849) 2025-08-14 21:14:53 -07:00
Stefan He
930fe467bd Support Triton FP8 Gemm can handle hidden_dim not divisible by 16 (#9093) 
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2025-08-12 21:21:55 -07:00
Elfie Guo
8723b4f146 Use FlashInfer's TRTLLM FP8 Blockscale GEMM (#8588) 2025-08-12 20:08:40 -07:00
Ying Sheng
c1d2061f97 Add initial support for gpt-oss (#8824) 2025-08-05 13:42:01 -07:00
mqhc2020
a562c8a35c [Dockerfile] Multi-arch support for ROCm (#7902) 
Co-authored-by: Lin, Soga <soga.lin@amd.com>
Co-authored-by: HaiShaw <hixiao@gmail.com>
 2025-07-14 06:13:09 +00:00
Lianmin Zheng
ce3a3e8783 Move multimodal processors into a separate folder (#7581) 2025-06-27 11:58:24 -07:00
valarLip
4c6675c4fc enable aiter fp8 blockscale quant (#7520) 2025-06-25 02:02:31 -07:00
JieXin Liang
5ca07eed90 [fix] fix DeepGEMM blackwell input quant & ut & fix style and log (#7247) 2025-06-16 11:45:54 -07:00
fzyzcjy
93cec4335f Support new DeepGEMM (#7172) 2025-06-13 23:00:17 -07:00
fzyzcjy
b4c41f7276 Refactor DeepGEMM integration (#7150) 2025-06-13 20:41:03 -07:00
fzyzcjy
5b1afa7814 Re-quantize DeepSeek model weights to support DeepGEMM new input format (#7156) 2025-06-13 15:57:45 -07:00
fzyzcjy
0f1dfa1efe Tiny add sanity checks for DeepGEMM inputs (#7157) 2025-06-13 14:36:27 -07:00
fzyzcjy
e3ec6bf4b6 Minor speed up block_quant_dequant (#6814) 2025-06-13 14:32:46 -07:00
Jianan Ji
5f91c82526 [Feature] Support Flashinfer fmha on Blackwell (#6930) 2025-06-06 12:57:50 -07:00
HAI
b819381fec AITER backend extension and workload optimizations (#6838) 
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Hubert Lu <Hubert.Lu@amd.com>
 2025-06-05 23:00:18 -07:00
fzyzcjy
35b65cf0ca Use deepgemm instead of triton for fused_qkv_a_proj_with_mqa (#6890) 2025-06-05 11:37:05 -07:00
Baizhou Zhang
6a47b73024 Remove contiguous before Flashinfer groupwise fp8 gemm (#6804) 2025-06-01 18:30:54 -07:00
ChangyiYang
485a023bd8 refactor apply_w8a8_block_fp8_linear in fp (#6545) 2025-05-29 00:15:11 -07:00
Baizhou Zhang
791b3bfabb [Feature] Support Flashinfer fp8 blockwise GEMM kernel on Blackwell (#6479) 2025-05-28 16:03:43 -07:00