chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
9,885 Commits 1 Branch 0 Tags
cc63c99f112f781605017db018fced686cdc94de
Commit Graph

287 Commits

Author SHA1 Message Date
Xiaoyu Zhang
1d3d42bda0 [opt kimi k2 1 / n] Add kimi k2 moe fused gate (#13287) 2025-11-15 17:14:19 +08:00
wangyxbh
5027739f2c [CPU] Use covt_e4m3_bf16 to optim BF16 to FP8 convert (#12191) 2025-11-14 17:36:51 +08:00
Binyao Jiang
9db3add319 Update GDN causal conv1d cuda kernel - prepare for new changes (#13188) 2025-11-13 14:09:47 -08:00
Fan Yin
2966367a31 [sgl-kernel] support custom fp8 flashmla kernel (#13087) 2025-11-13 12:45:21 -08:00
haoyangli-amd
ea10a9d165 [bug][rocm]fix qr when variable inp (#11609) 
Signed-off-by: Haoyang Li <lihaoyang0109@gmail.com>
 2025-11-11 01:43:48 -08:00
Xiaoyu Zhang
05559a4a90 Support hidden_dim % 4 == 0 in per_token_quant_fp8 (#12883) 2025-11-10 17:13:14 +08:00
huangtingwei
a5e5088dfb Fix errors of page head kernels in sgl-kernel for ROCm (#12604) 2025-11-10 15:15:50 +08:00
hlu1
b8ddc296f4 [sgl-kernel][Deepseek V3.2] Add row_starts to topk kernel (#12582) 
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
 2025-11-07 18:33:27 -08:00
jianan-gu
6e6009fb6b [CPU] Fix TP padding case with weight block size (#8243) 2025-11-07 03:24:48 +08:00
Lianmin Zheng
20315697f4 move all get_stream in sgl_kernel to c++ to reduce the launch overhead (#12521) 2025-11-02 13:15:05 -08:00
bingps
15ed27d7d4 [Fix] concat_mla_absorb_q_kernel fails for long inputs (#12453) 2025-11-02 11:52:06 -08:00
Xiaoyu Zhang
95191ebdca Migrate weak_ref_tensor to sgl-kernel (#12505) 2025-11-02 10:55:39 +08:00
Lianmin Zheng
c0652d907b Clean up sgl kernel (#12413) 
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
 2025-10-31 01:13:34 -07:00
AichenF
ed1044ac1b support cutlass fp4 kernel in sm120 (#11737) 2025-10-29 12:25:16 -07:00
Qi Yuhang
ce112c07fe [sgl-kernel][4/N]Support Expert Specialization Grouped GEMM (#12080) 2025-10-27 01:20:01 -07:00
Joonchen Liau
f7dc2f334b [sgl-kernel] feat: Support sm120 cutlass fp8 gemm kernel (#9403) 2025-10-26 23:45:45 -07:00
huangtingwei
3e6281d0aa [HiCache]Page head layout IO kernel (#11615) 2025-10-26 15:53:50 +08:00
Jinwu
13bf565d60 [2/N]Support DeepSeek-R1 w4a8 low latency deepep (#8464) 
Co-authored-by: Hank Han <hanhan7630@outlook.com>
Co-authored-by: Shangchuan Huang <2510421000@qq.com>
 2025-10-24 17:41:16 -07:00
blzheng
13fb8b5489 [CPU] Optimize FP16 decode_attention_cpu (#10652) 2025-10-22 21:39:51 -07:00
Fan Yin
23afdfd1c2 [sgl-kernel] support flashmla libtorch (#11717) 2025-10-21 21:17:50 -07:00
Serge Panev
2b1da821b5 [NVIDIA] Add new SMs support for Spark & Thor (#11287) 
Signed-off-by: Serge Panev <spanev@nvidia.com>
 2025-10-22 02:02:24 +08:00
hlu1
3b80232d06 [DeepseekV32] Add fast_topk_transform_ragged_fused kernel (#11815) 
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
 2025-10-19 17:13:39 -07:00
Johnny
252dc4e112 [NVIDIA] FA3/FA4 Fix (#11606) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-10-19 17:10:10 -07:00
Fan Yin
3289da5b41 [sgl-kernel] support hadamard (#11663) 2025-10-15 19:00:44 -07:00
Qi Yuhang
6c01844f45 [sgl-kernel][3/N]Support Expert Specialization Grouped GEMM (#11674) 2025-10-15 13:39:31 -07:00
Yineng Zhang
f792e3c561 Revert "[NVIDIA] BUMP FA3 (#11444)" (#11582) 2025-10-13 20:51:45 -07:00
Qi Yuhang
dc48c4c0e3 [sgl-kernel][2/N]Support Expert Specialization Grouped GEMM (#11534) 2025-10-13 16:24:48 -07:00
Johnny
b8c430f1ce [NVIDIA] BUMP FA3 (#11444) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
 2025-10-13 09:30:57 -07:00
Qi Yuhang
9a30914e94 [sgl-kernel][1/N]Support Expert Specialization Grouped GEMM (#11432) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: PGFLMG <1106310035@qq.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2025-10-12 20:19:21 -07:00
PGFLMG
8fdcd98efe [7/n] decouple quantization impl from vllm dependency - gguf kernel (#11019) 2025-10-11 14:04:57 -07:00
fzyzcjy
21337b22b9 Reland [1/2] Optimizations and refactors about quant kernel (#10312) 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-10-11 15:59:03 +08:00
Yuan Luo
4f42c8cd3e [sgl-kernel] Support float64 moe_sum_reduce cuda kernel (#11068) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-07 14:31:11 +00:00
DarkSharpness
e0b2d3eebe [Feature] Add a fast-topk to sgl-kernel for DeepSeek v3.2 (#11194) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-10-05 10:19:03 -07:00
Zhihao Zhang
24f7cb1ece [speculative decoding] rename lookahead to ngram (#11010) 
Co-authored-by: a4zhangfei <a4zhangfei@qq.com>
 2025-09-28 21:06:59 -07:00
Yuan Luo
42245551ef [sgl-kernel] Optimize concat_mla_k kernel (#10543) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: PGFLMG <1106310035@qq.com>
 2025-09-28 23:04:22 +08:00
Yuhao Yao
fe531d6f4e [Bug] Fix Issue#10215 (#10572) 2025-09-25 09:51:50 +08:00
Qi Yuhang
0f04a5f428 Optimize cutlass int8 gemm kernel for large M on SM89 Ada GPU (#10714) 2025-09-21 17:04:27 -07:00
Yuan Luo
616a3e20df [sgl-kernel] Support moe_sum_reduce cuda kernel (#10321) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2025-09-19 14:12:09 +08:00
Zhihao Zhang
e7bc600304 [Feature] Speculative decoding support lookahead (#9873) 
Co-authored-by: a4zhangfei <a4zhangfei@qq.com>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
 2025-09-18 16:42:41 -07:00
Qi Yuhang
9b876889b7 Update CUTLASS. Refine KernelSchedule for fp8 (grouped) gemm. (#10491) 2025-09-16 02:47:37 -07:00
fzyzcjy
3b25dc127a [1/2] Speed up trtllm_mla attention backend (>10% e2e) (#10473) 2025-09-15 11:53:21 -07:00
fzyzcjy
258d02c86d Fix correction bias undefined behavior for nvfp4 models (#10426) 2025-09-14 18:41:09 -07:00
Lianmin Zheng
c9ec4cae5b Fix the style of sgl kernel (#10398) 2025-09-12 22:20:21 -07:00
Hubert Lu
fe68c1486f Fix errors of hicache kernels in sgl-kernel for ROCm (#10339) 2025-09-11 14:54:34 -07:00
Yineng Zhang
6d55f60e77 Revert "[1/2] Optimizations and refactors about quant kernel (#9534)" (#10292) 2025-09-10 18:24:23 -07:00
Rain Jiang
2286e85e77 pass a_scale from fp8 quant result instead of hard code to 1.0f (#10241) 
Co-authored-by: Yichen Wang <yichen.wang@bytedance.com>
Co-authored-by: Jinwu Guo <641876696@qq.com>
 2025-09-10 12:56:05 -07:00
huangtingwei
5be8c2f7f7 Page first direct IO kernel (#10060) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-09-10 13:35:34 +08:00
Yi Zhang
8cbe1538ef Add mamba kernel (#10234) 2025-09-09 12:58:43 -07:00
blzheng
d1d4074c4e [CPU] Add gelu_and_mul kernel in sgl-kernel and add ut (#9300) 2025-09-08 23:23:13 -07:00
fzyzcjy
0096798ed6 [1/2] Speed up prefill mla attention (#10156) 2025-09-08 09:00:33 -07:00