chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
11,116 Commits 1 Branch 0 Tags
47cb9b9d116d1c5c112e894110ec94de80f2db6b
Commit Graph

75 Commits

Author SHA1 Message Date
Brayden Zhong
009eee85a0 CUTLASS FP8 Blockwise GEMM improvement of SM120 (#20887) 2026-03-22 17:55:54 +08:00
Xiaoyu Zhang
25e38216b6 [kernel slimming] Clean many useless sgl-kernel deprecated kernels (#20277) 2026-03-14 16:45:54 +08:00
Mohammad Miadh Angkad
f88acf8780 [JIT Kernel] Reland NVFP4 kernels to JIT (#20012) 2026-03-07 10:31:08 +08:00
Baizhou Zhang
51e5dc845a Revert "[Kernel Slimming] Migrate NVFP4 kernels to JIT" (#20005) 2026-03-05 19:40:00 -08:00
Mohammad Miadh Angkad
2bdd89a6cd [Kernel Slimming] Migrate NVFP4 kernels to JIT (#19437) 2026-03-05 15:22:28 +08:00
Xiaoyu Zhang
9dff933164 [Kernel Slimming] Remove sgl-kernel AOT marlin kernels (#19241) 2026-02-25 10:08:22 +08:00
blake-snc
0d30896015 fix(sgl-kernel): use >= 120 for SM12x CUDA kernel dispatch (#18750) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-16 00:44:47 +08:00
SoluMilken
07a24f1a38 update pre-commit config (#18860) 2026-02-16 00:18:31 +08:00
zhangxin81
e3021b65fe support smem in per_token_quant_fp8 kernel (#16725) 
Co-authored-by: zhangxin81 <969206500@qq.com>
 2026-02-02 17:18:50 +08:00
Serge Panev
e95668abc7 [NVIDIA] Fix CUDA arch requirement in nvfp4 cast (#12581) 
Signed-off-by: Serge Panev <spanev@nvidia.com>
Co-authored-by: Fan Yin <1106310035@qq.com>
 2026-01-21 20:21:11 -08:00
Xiaoyu Zhang
2ab3ed3e9e Fix sgl-kernel per_token_quant fp8 kernel scale shared_memory bug (#16886) 2026-01-13 23:22:05 +08:00
Xiaoyu Zhang
3de09aadbc Add new moe wna16 marlin gemm (#14122) 2025-12-01 23:07:53 +08:00
Xiaoyu Zhang
05559a4a90 Support hidden_dim % 4 == 0 in per_token_quant_fp8 (#12883) 2025-11-10 17:13:14 +08:00
Lianmin Zheng
20315697f4 move all get_stream in sgl_kernel to c++ to reduce the launch overhead (#12521) 2025-11-02 13:15:05 -08:00
AichenF
ed1044ac1b support cutlass fp4 kernel in sm120 (#11737) 2025-10-29 12:25:16 -07:00
Joonchen Liau
f7dc2f334b [sgl-kernel] feat: Support sm120 cutlass fp8 gemm kernel (#9403) 2025-10-26 23:45:45 -07:00
Serge Panev
2b1da821b5 [NVIDIA] Add new SMs support for Spark & Thor (#11287) 
Signed-off-by: Serge Panev <spanev@nvidia.com>
 2025-10-22 02:02:24 +08:00
fzyzcjy
21337b22b9 Reland [1/2] Optimizations and refactors about quant kernel (#10312) 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-10-11 15:59:03 +08:00
Qi Yuhang
0f04a5f428 Optimize cutlass int8 gemm kernel for large M on SM89 Ada GPU (#10714) 2025-09-21 17:04:27 -07:00
Yineng Zhang
6d55f60e77 Revert "[1/2] Optimizations and refactors about quant kernel (#9534)" (#10292) 2025-09-10 18:24:23 -07:00
Qi Yuhang
85ed8e0a5e Optimize nvfp4 block scaled gemm kernel when M is small. (#10101) 2025-09-06 22:31:00 -07:00
Jianying
dd1e268938 CUTLASS fp8 blockwise gemm support of sm120 (#9969) 2025-09-06 22:28:54 -07:00
DevashishLal-CB
dbb1235d58 [Fix] illegal sync based on undefined behaviour (#9620) 
Signed-off-by: Devashish Lal <devashish@rivosinc.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2025-09-06 11:54:48 +08:00
fzyzcjy
339f8eef09 [1/2] Optimizations and refactors about quant kernel (#9534) 2025-09-05 18:45:08 +08:00
Lianmin Zheng
d631290e32 Remove annoying warnings in sgl kernel build (#9905) 2025-09-02 20:18:25 -07:00
hlu1
1e85589dc5 Make fp4_quantize kernels work on sm103 (#9807) 
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
 2025-08-29 21:15:08 -07:00
Kaixi Hou
5c34b4f1c7 [NVIDIA] [2/N] Optimize silu_and_mul_scaled_fp4_grouped_quant perf (#9556) 2025-08-29 17:17:03 -07:00
hlu1
7a16db9bd9 Make sm100 fp8 kernels available on sm103 (#9789) 
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
 2025-08-28 23:47:29 -07:00
Kaixi Hou
e5638573c1 [NVIDA] [1/N] Nvfp4 Masked Gemm: Add quant op for the flashinfer grouped gemm (#9200) 2025-08-22 12:19:45 -07:00
Hubert Lu
c6c379ab31 [AMD] Reorganize hip-related header files in sgl-kernel (#9320) 2025-08-18 16:53:44 -07:00
jy-song-hub
4fc09e0df0 Fp4 MOE quant kernel optimization (#8777) 
Co-authored-by: Rain Jiang <96632942+rainj-me@users.noreply.github.com>
 2025-08-15 01:46:16 -07:00
Yuan Luo
432f2053dd [sgl-kernel] 1/N Refactor sglang cutlass 3x - gemm fp8 blockwise sm90 (#8913) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-08-14 10:55:54 -07:00
Peng Zhang
5aa1ebd242 [2/n]decouple quantization implementation from vLLM dependency (#8112) 
Co-authored-by: walker-ai <yiyun.wyt@antgroup.com>
Co-authored-by: leoneo <1320612015@qq.com>
 2025-08-14 03:19:03 -07:00
henryg
841810f227 [Perf] Tunings for SM100 FP8 CUTLASS kernel (#8818) 2025-08-13 21:59:22 -07:00
triple-mu
444013585d Fix typos and unify size(s)/stride(s) API calls (#8799) 2025-08-08 00:18:08 -07:00
Xiaoyu Zhang
f57d2dc162 [sgl-kernel] avoid per_token_quant_fp8.cu hardcode sm_count (#8738) 2025-08-04 12:55:57 +08:00
Stefan He
db7343c992 fix per token cuda kernel hidden dim cannot divide by 16 (#8543) 2025-08-01 09:27:18 -07:00
Peter Pan
6bdd27861b [Kimi K2] dsv3_router_gemm supports NUM_EXPERTS == 384 (#8013) 2025-08-01 22:01:24 +08:00
Xiaoyu Zhang
7a4309cc8a [sgl-kernel performace] fix fp8 quant kernels dispatch __nv_fp8_e4m3 bug to improve performance 10%-20% (#8499) 
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
 2025-07-29 23:31:54 +08:00
strgrb
fb4ce17de6 Fix per_token_group_quant_8bit when hidden_dim // group_size is not divided by 4. (#8449) 
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
 2025-07-28 01:32:46 -07:00
Yuan Luo
0c8dab9e67 [sgl-kernel] Opt per_token_quant_fp8 with warp reduce (#8130) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-07-23 21:22:59 +08:00
Baizhou Zhang
282eb59ff3 Add bf16 output option for dsv3_router_gemm kernel (#7999) 2025-07-20 09:49:37 +08:00
Qi Yuhang
6e92da8fca [Fix][Ready]Fix register spilling in cutlass nvfp4 gemm kernel on Blackwell (#8127) 2025-07-17 20:49:36 -07:00
likesen-alibaba
4a0d19198b Fix bug of deepseek-v3 under DP+EP mode with large batchsize/seqlen (#6449) 2025-07-10 01:19:56 -07:00
Baizhou Zhang
7248272ccc Add dsv3 router gemm kernel (#7627) 2025-06-29 23:31:55 -07:00
Ke Bao
04b35190e2 Add dsv3 fused a gemm to sgl-kernel (#7630) 2025-06-29 02:52:24 -07:00
AniZpZ
3eb4a800e8 Fix AWQ Dequant and Weight Loading of deepseek v2 (#6842) 2025-06-17 13:45:10 -07:00
fzyzcjy
5c66c4424f Support new DeepGEMM format in per token group quant (#7146) 2025-06-13 02:00:22 -07:00
Pavani Majety
eb38c7d1ca [1/2] Add Kernel support for Cutlass based Fused FP4 MoE (#6093) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-06-02 13:48:03 -07:00
HandH1998
4d643f6c7a [1/2] Support Qserve (#6457) 
Co-authored-by: yych0745 <1398089567@qq.com>
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
 2025-05-21 19:48:59 -07:00