chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
11,116 Commits 1 Branch 0 Tags
47cb9b9d116d1c5c112e894110ec94de80f2db6b
Commit Graph

50 Commits

Author SHA1 Message Date
Cao E
274581fb77 Add support for more batch sizes in cpu_graph_runner (#13881) 2026-03-19 09:50:56 -07:00
blzheng
cd22aa27a9 [CPU] Add FP8 Bmm support (#9744) 
Co-authored-by: Fan Yin <1106310035@qq.com>
 2026-03-18 22:19:48 -07:00
blzheng
c2b01bd2fc [CPU] fix bug in AVX512 implementation of flash_attn_softmax (#20220) 
Co-authored-by: Wu, Chunyuan <chunyuan.wu@intel.com>
 2026-03-18 22:18:47 -07:00
Ma Mingfei
687d9eb66f [CPU] Optimize image preprocessor performance for Qwen2VLImageProcessorFast (#15168) 2026-03-18 22:18:15 -07:00
Ma Mingfei
62d7454976 optimize conv3d used in patch embedding (#16040) 2026-03-18 22:17:53 -07:00
blzheng
cbea9f6909 [CPU] improve numa memory binding (#19666) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-18 22:15:50 -07:00
Zaili Wang
2f4babe32b [CPU] support LayerNorm with 3D shape (#15075) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-03-18 22:15:24 -07:00
blzheng
dc6aa26ce9 [CPU] Add mrope kernel for Qwen3-vl (#12531) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-03-18 22:12:48 -07:00
Cao E
4f0f6cd9d0 Add torch.compile support for qwen3-next on CPU (#12444) 2026-02-26 23:28:03 -08:00
jianan-gu
c35aa0238c [CPU][INT4] Add INT4 kernels for CPU (#8226) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-29 22:30:13 -08:00
Ma Mingfei
88f7759402 [CPU] optimize flash_attn_varlen_func (#15708) 2026-01-29 22:07:05 -08:00
jianan-gu
336dc4579e [CPU] Optimize Qwen3-next model on CPU (#12525) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
Co-authored-by: Fan Yin <1106310035@qq.com>
 2026-01-29 22:03:58 -08:00
blzheng
e27635a02d [CPU] Add 4D input support for ROPE in sgl-kernel (#9337) 2025-12-16 17:27:39 +08:00
blzheng
d16ff357db [CPU] Add Gemma3RMSNorm kernel in sgl-kernel and add ut (#9324) 2025-12-15 00:24:02 -08:00
Zaili Wang
d6bd2d1126 [CPU] layernorm & fused add-layernorm kernels (#14074) 2025-12-11 16:58:23 -08:00
Yibo Cai
7871593cc8 [cpu] Implement all gather/reduce for arm64 cpu (#12527) 2025-12-08 19:03:04 +08:00
blzheng
d257bf87b9 [CPU] add mamba fla kernels for Qwen3-next (#12324) 2025-12-06 14:16:23 +08:00
jianan-gu
70d2587324 [CPU] Optimize small oc GEMM for Qwen3-next on CPU (#12446) 
Co-authored-by: Zheng, Beilei <beilei.zheng@intel.com>
 2025-12-04 00:38:47 -08:00
Ma Mingfei
f90b400431 [CPU] add support for mamba causal conv1d for qwen3-next (#12309) 2025-12-04 13:41:42 +08:00
Ma Mingfei
46f21a5956 use faster covnersion from float8_e4m3fn to bfloat16 (#12316) 2025-12-04 12:34:05 +08:00
blzheng
974c562a25 [CPU] add fused_qkvzba_split_reshape_cat kernel for Qwen3-next (#12330) 2025-12-03 23:46:08 +08:00
Xuan Liao
c233e9d7a9 [CPU] Support chunk_gated_delta_rule kernel for Qwen3-Next (#12441) 2025-12-03 17:03:48 +08:00
Zaili Wang
0b0b2607ca [CPU] Apply uv as package manager (#14106) 2025-11-28 10:36:53 -08:00
YanbingJiang
acde21d8d5 Add fused_rmsnorm_gated_cpu kernel for CPU to support Qwen3-Next (#11577) 2025-11-21 01:33:31 +08:00
wangyxbh
5027739f2c [CPU] Use covt_e4m3_bf16 to optim BF16 to FP8 convert (#12191) 2025-11-14 17:36:51 +08:00
jianan-gu
6e6009fb6b [CPU] Fix TP padding case with weight block size (#8243) 2025-11-07 03:24:48 +08:00
blzheng
13fb8b5489 [CPU] Optimize FP16 decode_attention_cpu (#10652) 2025-10-22 21:39:51 -07:00
blzheng
d1d4074c4e [CPU] Add gelu_and_mul kernel in sgl-kernel and add ut (#9300) 2025-09-08 23:23:13 -07:00
Cao E
7577f0e40f Add graph runner support with torch compile on CPU (#7843) 2025-09-07 21:33:58 -07:00
Ma Mingfei
5ad296bda1 Optimize prefill performance on cpu backend (#8750) 2025-08-28 17:21:55 -07:00
Chunyuan WU
08f8f49016 [CPU][sgl-kernel] biased_grouped_topk: fix correction_bias dtype to float32 (#8212) 
Co-authored-by: jianan-gu <jianan.gu@intel.com>
Co-authored-by: YanbingJiang <yanbing.jiang@intel.com>
 2025-08-04 18:28:31 -07:00
YanbingJiang
1fe691a429 Fix FP8 block quantization when N or K is not multiples of 128 (#8648) 2025-08-01 15:57:19 -07:00
Chunyuan WU
ac80f4da57 [CPU] [FP8] set SGLANG_CPU_FP8_CVT_FTZ in CMakeLists.txt (#7885) 2025-07-09 01:53:53 -07:00
Chunyuan WU
128f16a817 [CPU]convert topk_weights to fp32 for INT8 and FP8 paths (for llama4) and fix LmHead weight pack (#7818) 2025-07-08 19:27:24 -07:00
Chunyuan WU
36cc3ffdc7 [CPU] [sgl-kernel] set dispatch key of initialize to CatchAll (#7734) 2025-07-02 22:39:24 -07:00
YanbingJiang
b044400dd3 Support non-contiguous query input for extend/decode attention (#7462) 2025-07-02 19:59:45 -07:00
Chunyuan WU
6005eceee3 [CPU] remove process_group from inputs of shm_allreduce and shm_allgather (#7486) 2025-06-30 21:54:11 -07:00
Chunyuan WU
c5131f7a2f [CPU] add c++ kernel to bind CPU cores and memory node (#7524) 2025-06-29 19:45:25 -07:00
Chunyuan WU
7eb47b0f3d [CPU] [BF16] Call fused_experts_cpu, weight_packed_linear and bmm_cpu kernel in DeepSeek model (#6641) 
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2025-06-25 01:43:33 -07:00
YanbingJiang
fcde67b016 CPU: map changes from developing branch in sgl-kernel (#6833) 
Co-authored-by: mingfeima <mingfei.ma@intel.com>
 2025-06-10 01:08:15 -07:00
jianan-gu
ff00895c46 Add CPU optimized kernels for topk and rope fusions (#6456) 2025-06-02 17:37:34 -07:00
Chunyuan WU
3ded6235c9 Add fp8 fused_experts kernel for CPU in sgl-kernel and add UT (#6404) 2025-05-23 02:01:55 -07:00
blzheng
4ba1eea83f Add fp8 qkv_proj_with_rope kernel for CPU in sgl-kernel and add UT (#6493) 2025-05-23 00:14:46 -07:00
blzheng
cfe48c5902 [CPU] Fix build issue (#6419) 2025-05-21 11:17:10 -07:00
YanbingJiang
32cc66efa5 Update extend/decode attention kernel for CPU in sgl-kernel and add UTs (#6405) 
Co-authored-by: mingfeima <mingfei.ma@intel.com>
 2025-05-19 21:23:17 -07:00
Chunyuan WU
5dd62c3a6f Add fp8 shared_expert kernel for CPU in sgl-kernel and add UT (#6339) 
Co-authored-by: Jiang, Yanbing <yanbing.jiang@intel.com>
Co-authored-by: mingfeima <mingfei.ma@intel.com>
 2025-05-18 12:42:15 -07:00
Chunyuan WU
fb4959b2c5 Add fp8 gemm kernel for CPU in sgl-kernel and add gemm UT (#6216) 
Co-authored-by: YanbingJiang <yanbing.jiang@intel.com>
Co-authored-by: mingfeima <mingfei.ma@intel.com>
 2025-05-15 09:10:40 -07:00
blzheng
0f75b907c6 [CPU] Add CMakeLists.txt for sgl-kernel (#6115) 2025-05-13 15:30:37 -07:00
applesaucethebun
2ce8793519 Add typo checker in pre-commit (#6179) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-05-11 12:55:00 +08:00
Ma Mingfei
a73c4df438 Add optimized native kernels in sgl-kernel (#5150) 
Co-authored-by: Chunyuan WU <chunyuan.wu@intel.com>
Co-authored-by: YanbingJiang <yanbing.jiang@intel.com>
Co-authored-by: blzheng <beilei.zheng@intel.com>
 2025-04-08 09:37:46 -07:00