chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
2,207 Commits 1 Branch 0 Tags
935cda944b82d4828e9efb6606cd449afa4cd71c
Commit Graph

69 Commits

Author SHA1 Message Date
Lianmin Zheng
935cda944b Misc clean up; Remove the support of jump forward (#4032) 2025-03-03 07:02:14 -08:00
Lianmin Zheng
110e006673 Reorganize python source files in sgl-kernel with multiple files (#4027) 2025-03-03 06:36:40 -08:00
Lianmin Zheng
6b45a21d16 Reorganize c++ source files in sgl-kernel with multiple folders (#4025) 2025-03-03 05:32:30 -08:00
Lianmin Zheng
66301e124f Improve code styles (#4021) 2025-03-03 03:20:23 -08:00
Lianmin Zheng
ac2387279e Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>
 2025-03-03 00:12:04 -08:00
Hubert Lu
9cf4077294 Enable custom AR for AMD GPUs and maintain it in sgl-kernel (#3406) 2025-03-02 15:19:06 -08:00
Chayenne
18bb216c28 Revert "[MOE] enable efficient moe_alignment multi-blocks execution (3x~6x)" (#3982) 2025-02-28 23:57:17 -08:00
yiakwy-xpu-ml-framework-team
1c96fa86cf [MOE] enable efficient moe_alignment multi-blocks execution (3x~6x) (#3613) 2025-02-27 19:42:48 -08:00
Xiaoyu Zhang
55a7ec388f use warp shuffle style reduce and flashinfer vectorize (#3628) 2025-02-19 20:53:51 +08:00
Baizhou Zhang
67fc595bb8 [Feature] Apply Cublas Grouped Gemm kernel (#3629) 2025-02-18 15:18:31 +08:00
Xiaoyu Zhang
3efbdf68b9 fix sgl-kernel codestyle (#3563) 2025-02-14 18:05:52 +08:00
Yineng Zhang
e082142519 chore: bump 0.0.3.post6 sgl-kernel (#3555) 2025-02-14 08:55:15 +08:00
Xiaoyu Zhang
f076328bb7 fix moe_align_kernel shm init not sync bug (#3534) 2025-02-13 16:47:00 +08:00
Yineng Zhang
4430c0a513 chore: bump 0.0.3.post5 sgl-kernel (#3530) 2025-02-13 01:51:46 +08:00
yizhang2077
640363ad20 support blockwise fp8 matmul kernel (#3267) 2025-02-13 01:49:33 +08:00
Yineng Zhang
b96e92e6e6 chore: bump 0.0.3.post4 sgl-kernel (#3523) 2025-02-12 17:28:36 +08:00
Xiaoyu Zhang
bb418ced80 optimize per token group quant fp8 (#3490) 2025-02-11 22:19:05 +08:00
Yineng Zhang
6239d0b2e7 chore: bump sgl-kernel v0.0.3.post3 (#3440) 2025-02-10 04:00:52 +08:00
Yineng Zhang
4cfd3add6d support version in sgl-kernel (#3439) 2025-02-10 03:49:52 +08:00
Yineng Zhang
f9905d59a8 support speculative decoding kernel in sgl-kernel (#3373) 
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
 2025-02-07 20:29:51 +08:00
Yineng Zhang
45c87e083f fix undefined symbol cudaGetDriverEntryPointByVersion (#3372) 2025-02-07 19:32:45 +08:00
Xiaoyu Zhang
cdae77b03d optimize moe_align_kernel cuda (#3347) 2025-02-07 00:53:46 +08:00
Yineng Zhang
adeee15204 fix sgl-kernel build failure on AMD (#3352) 2025-02-07 00:35:59 +08:00
Xiaoyu Zhang
ad3499858e clean moe align block kernel code and add acc test (#3332) 2025-02-06 16:42:36 +08:00
HAI
2c1a695ff1 ROCm: sgl-kernel enablement starting with sgl_moe_align_block (#3287) 2025-02-04 21:44:44 +08:00
Yineng Zhang
00fa7d0417 add copyright for sgl-kernel (#3270) 2025-02-03 21:34:44 +08:00
Yineng Zhang
c38b5fb4f4 update 3rdparty and rms norm for sgl-kernel (#3213) 2025-01-30 19:32:21 +08:00
Yineng Zhang
8a96f74988 chore: bump 0.0.3 for sgl-kernel (#3178) 
Co-authored-by: ispobock <ispobaoke@hotmail.com>
Co-authored-by: BBuf <35585791+BBuf@users.noreply.github.com>
Co-authored-by: HandH1998 <007aabbcc411@gmail.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: ByronHsu <byronhsu1230@gmail.com>
 2025-01-27 20:29:28 +08:00
Yineng Zhang
827aa8730b cleanup sgl-kernel kernels (#3175) 2025-01-27 19:11:01 +08:00
Byron Hsu
514f37c32b [kernel] Fix position ids in rope (#3173) 2025-01-27 17:09:51 +08:00
Byron Hsu
fb11a43981 [kernel] Integrate flashinfer's rope with higher precision and better perf (#3134) 2025-01-27 15:28:00 +08:00
HandH1998
82392da830 support w8a8 fp8 kernel with CUTLASS (#3047) 
Co-authored-by: yych0745 <1398089567@qq.com>
 2025-01-26 15:46:51 +08:00
Yineng Zhang
95f789adb0 minor: cleanup sgl-kernel (#3143) 2025-01-26 14:29:58 +08:00
yinfan98
9286740eff feat: refactor sgl-kernel and use TORCH_LIBRARY instead of PYBIND11_MODULE for custom ops (#3130) 
Co-authored-by: yinfan.1024 <yinfan.1024@bytedance.com>
Co-authored-by: yinfan98 <1106110035@qq.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-01-26 02:55:08 +08:00
yizhang2077
98522149ff mirror fix for custom allreduce (#3124) 2025-01-25 18:26:41 +08:00
Xiaoyu Zhang
5d9d15e70f support fp32 in sampling_scaling_penalties kernel (#3121) 2025-01-25 16:52:17 +08:00
Yineng Zhang
04f0b4cbef minor: update sgl-kernel setup (#3107) 2025-01-24 20:10:35 +08:00
Yineng Zhang
5de4051bcf feat: integrate sampling kernels into sgl-kernel (#3086) 
Co-authored-by: Zihao Ye <expye@outlook.com>
 2025-01-24 01:54:47 +08:00
Xiaoyu Zhang
f1b6861828 use flashinfer vec_dtypes in sgl_kernel (#3083) 2025-01-23 22:19:04 +08:00
Xiaoyu Zhang
ac2dc35d0e support lightning_attention_decode in sgl-kernel for MiniMax-Text-01 (#3030) 2025-01-23 15:29:20 +08:00
Yineng Zhang
806a3002c1 add notice about flashinfer in sgl-kernel (#3057) 2025-01-23 02:47:36 +08:00
Yineng Zhang
bf669606eb feat: integrate bmm_fp8 kernel into sgl-kernel (#3056) 2025-01-23 00:39:38 +08:00
Yineng Zhang
b2bd8f444c minor: update header and use pytest (#3054) 2025-01-22 23:45:18 +08:00
Yineng Zhang
9d9b482a39 feat: integrate activation kernels into sgl-kernel (#3053) 2025-01-22 23:25:45 +08:00
Yineng Zhang
7353fb9b97 feat: integrate norm kernels into sgl-kernel (#3052) 2025-01-22 21:32:48 +08:00
Yineng Zhang
9f8f2c7f74 update norm cu (#3048) 2025-01-22 18:58:44 +08:00
Ke Bao
0ac019f171 Support sm90 Int8 gemm (#3035) 2025-01-21 22:21:54 +08:00
Yineng Zhang
5a0d680a14 feat: add flashinfer as 3rdparty and use rmsnorm as example (#3033) 2025-01-21 20:44:49 +08:00
Byron Hsu
b5caa22dfb [kernel] port rope cuda kernel to sgl-kernel (#2993) 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-01-20 20:58:51 +08:00
Yineng Zhang
a69cb5cff7 cleanup unused header in sgl_kernel (#2986) 2025-01-20 00:44:49 +08:00