chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
4,993 Commits 1 Branch 0 Tags
065e523d7baa1beb67c9c6119c55f0ef203890fa
Commit Graph

53 Commits

Author SHA1 Message Date
PGFLMG
aa3eba8eb4 [sgl-kernel] misc: update deepgemm version for sgl-kernel (#9340) 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-08-27 12:01:30 -07:00
fzyzcjy
b1b3f0b38f Partially unify triton per token group quant kernels (#9485) 2025-08-23 02:07:31 -07:00
fzyzcjy
e85cb1ce9d Fix quant kernel test errors and benchmark wrong output speeds (#7604) 2025-08-21 03:48:41 -07:00
Stefan He
930fe467bd Support Triton FP8 Gemm can handle hidden_dim not divisible by 16 (#9093) 
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2025-08-12 21:21:55 -07:00
tql.99
3f2e315f6e optimize: reduce shulffle and quantization overhead in cutlass_moe sm90 (#8962) 
Co-authored-by: 戚余航 <qiyuhang@bytedance.com>
 2025-08-09 00:29:12 -07:00
fzyzcjy
403566bcca Remove assertions about per group quant fp8 (#8717) 2025-08-02 17:08:40 -07:00
fzyzcjy
b58c3c285e Support ue8m0 for triton quant kernel (#7603) 2025-07-27 13:04:35 -07:00
Hubert Lu
e50109f2ed [AMD] Remove vllm's scaled_fp8_quant and moe_sum when SGLANG_USE_AITER=1 (#7484) 2025-07-21 17:33:19 -07:00
YanbingJiang
b188a89a5d Fix CI xeon test with triton 3.3.1 (#8086) 2025-07-16 02:12:23 -07:00
Qi Yuhang
c268c11c71 [feat]Support fusion kernel for constructing quant input and scale factor for fp8_blockwise_scaled_grouped_mm (#8023) 2025-07-15 00:02:44 -07:00
likesen-alibaba
4a0d19198b Fix bug of deepseek-v3 under DP+EP mode with large batchsize/seqlen (#6449) 2025-07-10 01:19:56 -07:00
Lianmin Zheng
ce3a3e8783 Move multimodal processors into a separate folder (#7581) 2025-06-27 11:58:24 -07:00
JieXin Liang
5ca07eed90 [fix] fix DeepGEMM blackwell input quant & ut & fix style and log (#7247) 2025-06-16 11:45:54 -07:00
fzyzcjy
93cec4335f Support new DeepGEMM (#7172) 2025-06-13 23:00:17 -07:00
fzyzcjy
b4c41f7276 Refactor DeepGEMM integration (#7150) 2025-06-13 20:41:03 -07:00
fzyzcjy
bec3e48402 Support new DeepGEMM format in per token group quant (part 2: srt) (#7155) 2025-06-13 14:25:40 -07:00
ChangyiYang
485a023bd8 refactor apply_w8a8_block_fp8_linear in fp (#6545) 2025-05-29 00:15:11 -07:00
applesaucethebun
2ce8793519 Add typo checker in pre-commit (#6179) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-05-11 12:55:00 +08:00
lukec
acc816d8a2 DeepEP normal support deepgemm-contiguous (#5626) 
Co-authored-by: Yingyi Huang <yingyihuang2000@outlook.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Xuting Zhou <xutingz@nvidia.com>
Co-authored-by: ZhengHSI <zhenghsi@qq.com>
 2025-05-08 01:20:32 -07:00
Baizhou Zhang
73600673bb Clean logs for DeepSeek-V3 launching (#6079) 2025-05-07 18:54:50 -07:00
JieXin Liang
b70957fcf8 [refactor] slightly tidy fp8 module (#5993) 2025-05-07 17:28:24 -07:00
JieXin Liang
c2942907d5 [feature] enable pre compile jit deep_gemm (#5580) 2025-04-21 16:52:53 -07:00
JieXin Liang
99456bcacb [perf] introduce deep gemm group_gemm_masked as bmm (#5432) 2025-04-20 00:38:27 -07:00
fzyzcjy
613b197e57 Remove one kernel in per_tensor_quant_mla_fp8 (#5549) 2025-04-19 15:08:15 -07:00
strgrb
b2a189dd11 use sglang_per_token_group_quant_fp8 from sgl-kernel instead of trion kernel (#5473) 
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
 2025-04-18 00:05:24 -07:00
Lianmin Zheng
177320a582 Clean up imports (#5467) 2025-04-16 15:26:49 -07:00
Baizhou Zhang
f6772f1497 [Fix] Turn off DeepGEMM by default (#5263) 2025-04-14 17:45:44 -07:00
JieXin Liang
bdde237562 [perf] experimental enhance fp8 per-tensor quant (#5370) 2025-04-14 12:35:43 -07:00
Yineng Zhang
f58b929a51 chore: upgrade sgl-kernel 0.0.8.post3 (#5342) 2025-04-13 00:45:59 -07:00
Yineng Zhang
611720919d fix: use deepgemm only on hopper (#5310) 2025-04-11 20:48:24 -07:00
Yineng Zhang
6669d12707 feat: add DeepGEMM build warning (#5176) 
Co-authored-by: grimoire <streetyao@live.com>
 2025-04-08 21:16:23 -07:00
Lianmin Zheng
74e0ac1dbd Clean up import vllm in quantization/__init__.py (#4834) 2025-03-28 10:34:10 -07:00
lukec
4c56e5dbee Set deepgemm to the default value in the hopper architecture. (#4613) 2025-03-20 22:03:00 -07:00
strgrb
f9c53cbb42 Create col-major and tma-aligned x_scale for deep_gemm.gemm_fp8_fp8_bf16_nt (#4515) 
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
 2025-03-19 00:02:43 -07:00
Lianmin Zheng
45de89719c Revert "[XPU][CPU] Enable the native path of DeepSeek" (#4367) 2025-03-12 23:45:52 -07:00
Meng, Hengyu
71046fcd71 [XPU][CPU] Enable the native path of DeepSeek (#4086) 
Co-authored-by: Zhang, Liangang <liangang.zhang@intel.com>
 2025-03-12 22:26:29 -07:00
Lianmin Zheng
c76040e31b Support page size > 1 (#4356) 2025-03-12 22:22:39 -07:00
AniZpZ
85ef7f64e4 [FIX] fix incorrect output when enable both deepgemm and torch compile (#4359) 
Co-authored-by: xuyongfei.xyf <xuyongfei.xyf@antgroup.com>
 2025-03-12 21:34:09 -07:00
Yineng Zhang
d1da58e275 unify is_cuda and is_hip (#4321) 2025-03-11 18:12:56 -07:00
lukec
dce303e279 linear support deepgemm (#4199) 
Co-authored-by: yinfan98 <1106310035@qq.com>
 2025-03-11 00:38:37 -07:00
HandH1998
0dd6cda288 Apply sgl w8a8 fp8 kernel (#3148) 2025-03-09 00:03:32 -08:00
Yineng Zhang
5f1a485d9e Revert "[ROCm] Use tl.range() in block GEMM kernels with `num_stage… (#3632) 2025-02-17 18:01:21 +08:00
Wen-Heng (Jack) Chung
03caefeb51 [ROCm] Use tl.range() in block GEMM kernels with num_stages set by host. (#3535) 
Co-authored-by: HAI <hixiao@gmail.com>
 2025-02-16 01:40:38 -08:00
yizhang2077
98eecbda54 integrate blockwise fp8 kernel (#3529) 2025-02-13 04:39:33 +08:00
Xiaoyu Zhang
45e3a7bc41 use sgl_per_token_group_quant_fp8 kernel (#3493) 2025-02-12 18:40:42 +08:00
Wen-Heng (Jack) Chung
32de54ed1a [ROCm] Fix fp8 unrolledx4 matmul kernel. (#3325) 
Co-authored-by: HAI <hixiao@gmail.com>
 2025-02-05 18:44:15 -08:00
Wen-Heng (Jack) Chung
7ab84948d8 [ROCm] Logic to decide whether to used manually unrolled kernel. (#3306) 2025-02-04 19:12:20 -08:00
Wen-Heng (Jack) Chung
c2723a42a5 [ROCm] Manually unroll _w8a8_block_fp8_matmul kernel on AMD GPU. (#3299) 2025-02-05 07:15:40 +08:00
HandH1998
afa0341e57 Update Triton configs for block fp8 kernels (#2641) 2024-12-29 22:53:47 +08:00
HAI
30828e7192 AMD: set weights and scaling numbers properly for block FP8 (#2637) 2024-12-29 03:23:39 -08:00