chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
9,885 Commits 1 Branch 0 Tags
cc63c99f112f781605017db018fced686cdc94de
Commit Graph

71 Commits

Author SHA1 Message Date
blake-snc
0d30896015 fix(sgl-kernel): use >= 120 for SM12x CUDA kernel dispatch (#18750) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-16 00:44:47 +08:00
Bruce-x-1997
793c96c3d2 [perf]optimize w4afp8 kernel on deepseek-v3-0324 (#12921) 
Signed-off-by: bruce.xu <bruce.x@gmicloud.ai>
 2025-12-18 18:13:22 +08:00
Kevin_Xiong
4792d1f452 [sgl-kernel][1/2] Fused qk_norm_rope for GLM4.6 (#15141) 2025-12-18 17:07:04 +08:00
Xiaoyu Zhang
56d12b4aea Fix warp illegal instruction in kimi k2 thinking PCG (#15306) 2025-12-18 16:58:23 +08:00
zyl_keep_moving
a9ce1623cd [kernel][moe] add moe topk fast (#13969) 
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2025-12-14 22:26:40 +08:00
Xiaoyu Zhang
c5947ecd85 Opt moe align block size kernel (#14133) 2025-12-02 19:13:55 +08:00
Xiaoyu Zhang
3de09aadbc Add new moe wna16 marlin gemm (#14122) 2025-12-01 23:07:53 +08:00
Yuan Luo
e12c78aab6 [sgl-kernel][1/2] Fused qk_norm_rope for Qwen3-MoE (#14036) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-28 12:25:15 +08:00
Roger Young
e72cf13693 Support moe topk sigmoid kernel (#13049) 
Co-authored-by: xuebi <xuebi@minimaxi.com>
 2025-11-20 00:24:37 +08:00
Xiaoyu Zhang
820e13c9c1 [opt kimi k2 3/n] opt kimi_k2 moe_fused_gate kernel (#13374) 2025-11-18 15:36:31 +08:00
Xiaoyu Zhang
1d3d42bda0 [opt kimi k2 1 / n] Add kimi k2 moe fused gate (#13287) 2025-11-15 17:14:19 +08:00
Lianmin Zheng
c0652d907b Clean up sgl kernel (#12413) 
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
 2025-10-31 01:13:34 -07:00
AichenF
ed1044ac1b support cutlass fp4 kernel in sm120 (#11737) 2025-10-29 12:25:16 -07:00
Jinwu
13bf565d60 [2/N]Support DeepSeek-R1 w4a8 low latency deepep (#8464) 
Co-authored-by: Hank Han <hanhan7630@outlook.com>
Co-authored-by: Shangchuan Huang <2510421000@qq.com>
 2025-10-24 17:41:16 -07:00
PGFLMG
8fdcd98efe [7/n] decouple quantization impl from vllm dependency - gguf kernel (#11019) 2025-10-11 14:04:57 -07:00
Yuan Luo
4f42c8cd3e [sgl-kernel] Support float64 moe_sum_reduce cuda kernel (#11068) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-07 14:31:11 +00:00
Yuan Luo
616a3e20df [sgl-kernel] Support moe_sum_reduce cuda kernel (#10321) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2025-09-19 14:12:09 +08:00
Qi Yuhang
9b876889b7 Update CUTLASS. Refine KernelSchedule for fp8 (grouped) gemm. (#10491) 2025-09-16 02:47:37 -07:00
fzyzcjy
258d02c86d Fix correction bias undefined behavior for nvfp4 models (#10426) 2025-09-14 18:41:09 -07:00
Rain Jiang
2286e85e77 pass a_scale from fp8 quant result instead of hard code to 1.0f (#10241) 
Co-authored-by: Yichen Wang <yichen.wang@bytedance.com>
Co-authored-by: Jinwu Guo <641876696@qq.com>
 2025-09-10 12:56:05 -07:00
Yuhao Yao
ee0b3c5bad [1/N][Bug] Fix w4afp8 MoE NaN issue (sgl-kernel, fixed) (#10108) 2025-09-07 21:39:07 -07:00
hlu1
5f1eb20484 [chore] Remove unused ep_moe cuda kernels (#9956) 2025-09-06 01:35:50 -07:00
Yineng Zhang
0e78c63c0e Revert "[1/N][Bug] Fix w4afp8 MoE NaN issue (sgl-kernel) (#9953)" (#10097) 2025-09-05 19:57:53 -07:00
Yuhao Yao
f78b7fd16d [1/N][Bug] Fix w4afp8 MoE NaN issue (sgl-kernel) (#9953) 2025-09-03 18:28:27 +08:00
chenxj
d4a938417d [feat] Support tp mode for DeepSeek-R1-W4AFP8 (#8118) 
Co-authored-by: yuhyao <827623970@qq.com>
 2025-09-01 22:17:26 -07:00
hlu1
7a16db9bd9 Make sm100 fp8 kernels available on sm103 (#9789) 
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
 2025-08-28 23:47:29 -07:00
Rain Jiang
6b39f9cf8c Support compile sgl-kernel on cuda 13.0 (#9721) 2025-08-28 10:18:03 -07:00
PGFLMG
aa3eba8eb4 [sgl-kernel] misc: update deepgemm version for sgl-kernel (#9340) 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-08-27 12:01:30 -07:00
Rain Jiang
79e6a8a6ac support cuda 13.0 and trtllm kernel by Aug 25 2025 (#9495) 2025-08-26 23:13:27 -07:00
Qi Yuhang
fda4792620 Update CUTLASS 4.2 & Enable K-Major Scale Factor for SM90 FP8 Blockwise Group GEMM (#9559) 2025-08-24 23:24:43 -07:00
kousakawang
5fd311d33e [code clean] add H20 cutlass groupGemm default config (#9333) 
Co-authored-by: wanghanpei <wanghanpei@bytedance.com>
 2025-08-21 19:23:29 -07:00
Hubert Lu
c6c379ab31 [AMD] Reorganize hip-related header files in sgl-kernel (#9320) 2025-08-18 16:53:44 -07:00
kousakawang
0fc54b971e [fix]: fix cutlass moe ut and and Opt H20 cutlass groupGemm performance (#9272) 
Co-authored-by: wanghanpei <wanghanpei@bytedance.com>
 2025-08-17 13:09:49 -07:00
Peng Zhang
5aa1ebd242 [2/n]decouple quantization implementation from vLLM dependency (#8112) 
Co-authored-by: walker-ai <yiyun.wyt@antgroup.com>
Co-authored-by: leoneo <1320612015@qq.com>
 2025-08-14 03:19:03 -07:00
Trevor Morris
13c48dcf88 [1/2][resubmit again] sgl-kernel: Fuse routed scaling factor into moe_fused_gate (#9088) 2025-08-12 20:12:38 -07:00
Yineng Zhang
dd949ace23 Revert "[1/2][resubmit] sgl-kernel: Fuse routed scaling factor into m… (#9035) 2025-08-10 17:34:54 -07:00
Trevor Morris
591c232f7c [1/2][resubmit] sgl-kernel: Fuse routed scaling factor into moe_fused_gate (select_experts) (#8770) 2025-08-08 17:55:06 -07:00
Qi Yuhang
d9def43dcd [Perf]Use Cooperative Schedule for H100 & H200 & H800 in fp8_blockwise_scaled_grouped_mm (#8722) 2025-08-02 21:13:47 -07:00
Liangsheng Yin
f9f0138f80 Revert "[1/2] sgl-kernel: Fuse routed scaling factor into select_experts" (#8706) 2025-08-02 20:14:30 +08:00
Trevor Morris
f642524fd9 [1/2] sgl-kernel: Fuse routed scaling factor into select_experts (#8364) 2025-08-01 18:14:24 -07:00
Cheng Wan
a5f5ab4030 update sgl-kernel for EP: kernel part (#8514) 
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
 2025-07-30 22:19:55 -07:00
Qi Yuhang
9b9e82539b [Fix]Fix index oob in get_group_gemm_starts kernel. (#8564) 2025-07-30 19:49:35 -07:00
Yuan Luo
3bdcdd134b [Hot-Fix] moe_aligned_block_size CI failed in AMD (#8461) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: JieXin Liang <Alcanderian@users.noreply.github.com>
 2025-07-31 00:28:32 +08:00
Xiaoyu Zhang
2262369905 Revert "[kernel] opt moe align block kernel by block/warp scan algorithm" (#8457) 2025-07-28 01:35:43 -07:00
Yuan Luo
af1cc8fe2d [kernel] opt moe align block kernel by block/warp scan algorithm (#7884) 2025-07-17 19:33:02 +08:00
Ke Bao
a3398d8478 Optimize moe align block size kernel (#7794) 2025-07-07 09:20:30 +08:00
Mick
c797322280 fix: fix apply_shuffle_mul_sum (#7444) 2025-07-04 23:23:30 -07:00
Qi Yuhang
8e9fb43d82 Optimize Hopper CUTLASS FP8 Blockwise Grouped GEMM Kernel in Small K Scenario (#7782) 2025-07-04 22:25:49 -07:00
SijiaYang
da3890e82a [1/n]: add cutlass W4A8 moe kernel for hopper architecture (#7772) 
Signed-off-by: yangsijia.614 <yangsijia.614@bytedance.com>
Co-authored-by: yicwang <yichen.wang@bytedance.com>
 2025-07-04 20:50:12 -07:00
Yi Zhang
2998c4bdf4 [optimize] fuse renormalize into moe_topk_softmax (#7744) 
Co-authored-by: ispobock <ispobaoke@gmail.com>
 2025-07-03 12:42:44 -07:00