chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
11,116 Commits 1 Branch 0 Tags
47cb9b9d116d1c5c112e894110ec94de80f2db6b
Commit Graph

308 Commits

Author SHA1 Message Date
ThomasX
3a5928ef53 feat: add page-aligned index validation for CP HiCache direct transfers 
Add validate_page_aligned_token_indices utility and apply it across
HiCache write/load paths, NSA indexer transfers, and CUDA direct copy
kernels to reject malformed (partial, misaligned, non-contiguous) page
groups before they reach native transfer code. Also validate supported
CP HiCache backend/layout combinations at server startup.
 2026-05-10 02:02:54 +08:00
Brayden Zhong
009eee85a0 CUTLASS FP8 Blockwise GEMM improvement of SM120 (#20887) 2026-03-22 17:55:54 +08:00
Cao E
274581fb77 Add support for more batch sizes in cpu_graph_runner (#13881) 2026-03-19 09:50:56 -07:00
Qi Yuhang
cb8105fe28 [sgl-kernel][6/7]Support Expert Specialization Grouped GEMM (#15471) 2026-03-19 15:39:52 +08:00
blzheng
cd22aa27a9 [CPU] Add FP8 Bmm support (#9744) 
Co-authored-by: Fan Yin <1106310035@qq.com>
 2026-03-18 22:19:48 -07:00
blzheng
c2b01bd2fc [CPU] fix bug in AVX512 implementation of flash_attn_softmax (#20220) 
Co-authored-by: Wu, Chunyuan <chunyuan.wu@intel.com>
 2026-03-18 22:18:47 -07:00
Ma Mingfei
687d9eb66f [CPU] Optimize image preprocessor performance for Qwen2VLImageProcessorFast (#15168) 2026-03-18 22:18:15 -07:00
Ma Mingfei
62d7454976 optimize conv3d used in patch embedding (#16040) 2026-03-18 22:17:53 -07:00
blzheng
cbea9f6909 [CPU] improve numa memory binding (#19666) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-18 22:15:50 -07:00
Zaili Wang
2f4babe32b [CPU] support LayerNorm with 3D shape (#15075) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-03-18 22:15:24 -07:00
blzheng
dc6aa26ce9 [CPU] Add mrope kernel for Qwen3-vl (#12531) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-03-18 22:12:48 -07:00
Xiaoyu Zhang
25e38216b6 [kernel slimming] Clean many useless sgl-kernel deprecated kernels (#20277) 2026-03-14 16:45:54 +08:00
Mohammad Miadh Angkad
f88acf8780 [JIT Kernel] Reland NVFP4 kernels to JIT (#20012) 2026-03-07 10:31:08 +08:00
Baizhou Zhang
51e5dc845a Revert "[Kernel Slimming] Migrate NVFP4 kernels to JIT" (#20005) 2026-03-05 19:40:00 -08:00
Mohammad Miadh Angkad
2bdd89a6cd [Kernel Slimming] Migrate NVFP4 kernels to JIT (#19437) 2026-03-05 15:22:28 +08:00
huangtingwei
36dc973cbf [HiCache] refactor page_first_direct io kernel (#18113) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
 2026-02-27 11:43:14 -08:00
Cao E
4f0f6cd9d0 Add torch.compile support for qwen3-next on CPU (#12444) 2026-02-26 23:28:03 -08:00
Xiaoyu Zhang
054bd71086 [sgl-kernel slimming] remove sgl-kernel moe-wna16-marlin (#19379) 2026-02-27 09:17:46 +08:00
Qi Yuhang
88ad3b894a [sgl-kernel][Feat][B200][2/N] Support MXFP8 Grouped GEMM in Blackwell (#14640) 
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2026-02-26 11:23:37 +08:00
pansicheng
2ad475b4ed use flashinfer.sampling (#18696) 2026-02-26 10:02:38 +08:00
Xiaoyu Zhang
9dff933164 [Kernel Slimming] Remove sgl-kernel AOT marlin kernels (#19241) 2026-02-25 10:08:22 +08:00
blake-snc
0d30896015 fix(sgl-kernel): use >= 120 for SM12x CUDA kernel dispatch (#18750) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-16 00:44:47 +08:00
SoluMilken
07a24f1a38 update pre-commit config (#18860) 2026-02-16 00:18:31 +08:00
Xiaoyu Zhang
c29394e3c8 [kernel slimming] Move fast_hadamard_transform to jit_kernel (#18475) 2026-02-14 23:06:21 +08:00
Xiaoyu Zhang
9e9e949261 speed up sgl-kernel build (#18586) 2026-02-12 23:43:22 +08:00
Baizhou Zhang
2d38b8aca0 Revert "[sgl-kernel] upgrade deepgemm" (#18562) 2026-02-11 01:17:40 +08:00
Xiaoyu Zhang
bec7fe9e65 [sgl-kernel] upgrade deepgemm (#18362) 2026-02-10 21:31:30 +08:00
zhangxin81
e3021b65fe support smem in per_token_quant_fp8 kernel (#16725) 
Co-authored-by: zhangxin81 <969206500@qq.com>
 2026-02-02 17:18:50 +08:00
Yuan Luo
afebb7ab78 Optimize custom-all-reduce (#17674) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-02-01 18:59:31 +08:00
Yifan Cui
45fe51a28e Reduce topk kernel shared memory from 128KB to 32KB for better occupancy (#17747) 
Co-authored-by: Claude <noreply@anthropic.com>
 2026-01-30 21:42:21 -08:00
jianan-gu
c35aa0238c [CPU][INT4] Add INT4 kernels for CPU (#8226) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-29 22:30:13 -08:00
Ma Mingfei
88f7759402 [CPU] optimize flash_attn_varlen_func (#15708) 2026-01-29 22:07:05 -08:00
jianan-gu
336dc4579e [CPU] Optimize Qwen3-next model on CPU (#12525) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
Co-authored-by: Fan Yin <1106310035@qq.com>
 2026-01-29 22:03:58 -08:00
Xiaoyu Zhang
fb74e43707 [Diffusion] Delete sgl-kernel outdated time_embedding kernel (#17278) 2026-01-28 14:18:53 +08:00
R0CKSTAR
628ab5d57b [MUSA][2/N] sgl-kernel build (#17053) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2026-01-23 14:41:47 -08:00
Serge Panev
e95668abc7 [NVIDIA] Fix CUDA arch requirement in nvfp4 cast (#12581) 
Signed-off-by: Serge Panev <spanev@nvidia.com>
Co-authored-by: Fan Yin <1106310035@qq.com>
 2026-01-21 20:21:11 -08:00
Binyao Jiang
38c233fd04 [Piecewise] Support PCG weak_ref_tensor cuda kernel on AMD (#17291) 2026-01-20 14:05:32 -08:00
Michael
53609e5e5b Revert "[Diffusion] Move diffusion time embedding to jit kernel" (#17257) 2026-01-17 21:29:22 +08:00
Xiaoyu Zhang
2cdd4370bc [Diffusion] Move diffusion time embedding to jit kernel (#16879) 2026-01-17 12:21:22 +08:00
Lianmin Zheng
a4825ed588 Fix kernel type annotations for fp8 quant and logging (#16994) 2026-01-13 18:14:32 -08:00
Xiaoyu Zhang
2ab3ed3e9e Fix sgl-kernel per_token_quant fp8 kernel scale shared_memory bug (#16886) 2026-01-13 23:22:05 +08:00
Hubert Lu
8716589826 [AMD][Diffusion] support timestep embedding kernel for AMD GPUs (#16766) 2026-01-12 22:17:07 -08:00
Hubert Lu
51e2eaa458 [AMD] Support fast_topk kernels in sgl-kernel (#15172) 2025-12-19 22:19:09 -08:00
66RING
46be74b4b4 [diffusion] kernel: timestep embedding kernel implementation (#12995) 
Co-authored-by: 戚余航 <qiyuhang@bytedance.com>
Co-authored-by: Qi Yuhang <45795032+HydraQYH@users.noreply.github.com>
 2025-12-19 20:59:50 +08:00
sunxxuns
f2d64e6782 [amd] Add deterministic all-reduce kernel for AMD (ROCm) (#15340) 
Co-authored-by: Thomas Wang <1am9trash@gmail.com>
 2025-12-18 23:36:03 -08:00
Bruce-x-1997
793c96c3d2 [perf]optimize w4afp8 kernel on deepseek-v3-0324 (#12921) 
Signed-off-by: bruce.xu <bruce.x@gmicloud.ai>
 2025-12-18 18:13:22 +08:00
Kevin_Xiong
4792d1f452 [sgl-kernel][1/2] Fused qk_norm_rope for GLM4.6 (#15141) 2025-12-18 17:07:04 +08:00
Xiaoyu Zhang
56d12b4aea Fix warp illegal instruction in kimi k2 thinking PCG (#15306) 2025-12-18 16:58:23 +08:00
blzheng
e27635a02d [CPU] Add 4D input support for ROPE in sgl-kernel (#9337) 2025-12-16 17:27:39 +08:00
blzheng
d16ff357db [CPU] Add Gemma3RMSNorm kernel in sgl-kernel and add ut (#9324) 2025-12-15 00:24:02 -08:00