chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
3,213 Commits 1 Branch 0 Tags
e9bebafb190dadeaa76e6e5b224fef552a9df139
Commit Graph

130 Commits

Author SHA1 Message Date
applesaucethebun
2ce8793519 Add typo checker in pre-commit (#6179) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-05-11 12:55:00 +08:00
lukec
acc816d8a2 DeepEP normal support deepgemm-contiguous (#5626) 
Co-authored-by: Yingyi Huang <yingyihuang2000@outlook.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Xuting Zhou <xutingz@nvidia.com>
Co-authored-by: ZhengHSI <zhenghsi@qq.com>
 2025-05-08 01:20:32 -07:00
Baizhou Zhang
cfca4e0ed2 adding Triton configs for DeepSeekV3 FusedMoE kernel on Blackwell (#6111) 2025-05-07 23:39:10 -07:00
Baizhou Zhang
73600673bb Clean logs for DeepSeek-V3 launching (#6079) 2025-05-07 18:54:50 -07:00
JieXin Liang
b70957fcf8 [refactor] slightly tidy fp8 module (#5993) 2025-05-07 17:28:24 -07:00
JieXin Liang
8ebde73f7d [perf] H100 DeepSeek-V3 fused moe tuned config (#5998) 2025-05-03 14:02:26 -07:00
lambert0312
1698e94e67 Add A800 fused moe config for qwen3 235b (#5900) 2025-04-29 20:18:11 -07:00
ybyang
cc4a80caf6 [PD] Fix Assertion failed: /DeepEP/csrc/kernels/internode.cu:483, condition: ibgda_get_state()->num_rc_per_pe >= num_channels #134 (#5830) 2025-04-29 19:38:54 -07:00
lambert0312
91dda4cd06 Add A800 fused moe config for qwen3 30b (#5880) 2025-04-29 02:02:24 -07:00
XinyuanTong
8465f035d1 Add qwen3 30b fused moe config (#5859) 2025-04-29 00:24:00 -07:00
Xiaoyu Zhang
1cc326032d simplify fused_moe config logging (#5801) 2025-04-28 17:04:54 -07:00
Yi Zhang
a0251a3fd6 add fused moe config for qwen3moe fp8/bf16 (#5849) 2025-04-28 11:55:52 -07:00
HAI
d364b9b0f2 ROCm: update AITER (#5816) 2025-04-28 11:01:20 -07:00
JiLi
d73ddeb196 feat: Add fused moe triton config for qwen3-30b-fp8 moe on h20 (#5850) 2025-04-28 10:49:33 -07:00
ybyang
74cb12a878 [config] qwen3moe_tune_h20 fp8 tp4 (#5846) 2025-04-28 10:21:06 -07:00
yhyang201
92ab0a2055 feat: Add fused moe triton config for qwen3bf16 moe on h20 (#5839) 2025-04-28 09:30:59 -07:00
XinyuanTong
0045f4b2af feat: Add fused moe triton config for qwen3 moe on h100 (#5833) 2025-04-28 08:37:13 -07:00
Lianmin Zheng
621e96bf9b [CI] Fix ci tests (#5769) 2025-04-27 07:18:10 -07:00
lambert0312
a086a11305 Use sgl-kernel sgl_per_token_group_quant_int8 (#4971) 2025-04-26 20:19:42 -07:00
saltyfish66
133ded039a perf: update H20 fused_moe_triton kernel config to get higher throughput during prefilling (#5716) 2025-04-26 18:15:07 -07:00
Xiaoyu Zhang
18ce468d56 update triton 3.2.0 h200 fused moe triton config and add warning about triton fused_moe_kernel performance degradation due to different Triton versions. (#5740) 2025-04-25 16:24:59 -07:00
saltyfish66
2aa3f5e2d0 [feature] Add H20 fp8_w8a8 FusedMoE config for --n-share-experts-fusion=16 (#5641) 
Co-authored-by: yuethe <yuethe@tencent.com>
 2025-04-22 09:33:13 -07:00
fzyzcjy
463d4b7400 Fix DeepEP cannot run on latest master (#5567) 2025-04-20 14:19:42 -07:00
Xiaoyu Zhang
d58e354472 simplify the control logic for using shared experts fusion (#5504) 2025-04-19 13:17:35 -07:00
fzyzcjy
1e0806f30b Fix DeepGEMM masked cannot be run on groups not being multiple or 4 (#5340) 2025-04-18 22:38:07 -07:00
Xiaoyu Zhang
bed05878f6 fix kimi vl running bug after rebase main (#5461) 2025-04-18 00:17:34 -07:00
Lianmin Zheng
177320a582 Clean up imports (#5467) 2025-04-16 15:26:49 -07:00
Ximingwang-09
2dd6489468 Add H20 dtype fp8_w8a8 shared experts fused MoE kernel tuning configs for DeepSeek V3/R1 (#5291) 
Co-authored-by: ximing.wxm <ximing.wxm@antgroup.com>
 2025-04-14 18:40:31 -07:00
lambert0312
61e7c4dd21 Add A800 shared experts fused MoE kernel tuning configs for DeepSeek V3/R1 (#5368) 2025-04-14 18:39:44 -07:00
Xiaoyu Zhang
38076dea84 apply fused moe gate in ds v3/r1 (#5371) 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-04-14 16:24:26 -07:00
Xiaoyu Zhang
3e4794aad8 refine fused_moe tuning docs (#5294) 2025-04-12 10:01:13 -07:00
Xiaoyu Zhang
690ec20587 Delete python/sglang/srt/layers/moe/fused_moe_triton/configs/E=257,N=… (#5321) 2025-04-12 10:00:03 -07:00
Xiaoyu Zhang
60bcbf2a35 remove moe_align_block_size torch.zeros in small batch/expert mode (#5298) 2025-04-11 12:13:55 -07:00
Xiaoyu Zhang
f730362ee2 reduce moe_align_block_size_kernel small batch mode overhead (#5086) 2025-04-09 17:59:35 -07:00
Zhaoyang Hao
456b008bd8 Add H20 dtype fp8_w8a8 fused MoE kernel tuning configs for DeepSeek V3/R1 (#5196) 2025-04-09 11:54:36 -07:00
HandH1998
4065248214 Support Llama4 fp8 inference (#5194) 
Co-authored-by: laixinn <xielx@shanghaitech.edu.cn>
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: zhyncs <me@zhyncs.com>
 2025-04-09 20:14:34 +08:00
Cheng Wan
76c48a0913 [DeepEP] fix: import buffer error (#5179) 2025-04-08 22:12:14 -07:00
Jinyan Chen
bc3f6db2dd [Fix] DeepEP Compatibility with Low Latency (#5068) 
Co-authored-by: ch-wan <cwan39@gatech.edu>
 2025-04-08 20:31:31 -07:00
kk
5a144a8ab9 Fix run time error in ROCm platform (#5147) 
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: root <root@dell300x-pla-t10-17.pla.dcgpu>
 2025-04-07 22:49:40 -07:00
Hubert Lu
afb752bcbe [AMD] Fix missing per_token_group_quant_fp8 for ROCm (#5140) 2025-04-07 22:38:25 -07:00
Chang Su
f04c80dc42 Add Llama4 support (#5092) 
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
Co-authored-by: ispobock <ispobaoke@163.com>
 2025-04-07 00:29:36 -07:00
Xiaoyu Zhang
924ca7c92c Add DeepSeek V3/R1 shared experts fusion (#4918) 2025-04-04 01:59:29 -07:00
fzyzcjy
6ff9c6a5e7 Cleanup unused resources after DeepEP operation (#4996) 2025-04-04 00:36:04 -07:00
fzyzcjy
77e929a1a2 Support async DeepEP by splitting into two stages (#4995) 2025-04-04 00:32:27 -07:00
fzyzcjy
febe21ce03 Small refactor DeepEPDispatcher into subclasses (#4994) 2025-04-04 00:24:18 -07:00
Tommy Yang
31035dda44 Add H20 fused MoE kernel tuning configs for DeepSeek V3/R1 (#5057) 2025-04-03 22:14:28 -07:00
AniZpZ
d95269f9b3 [2/3] fix dsv3 awq issue (#4625) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: laixinn <xielx@shanghaitech.edu.cn>
 2025-04-03 17:36:39 -07:00
fzyzcjy
8e10fec9a8 Small refactor DeepEPMode to clean up code a bit (#4992) 2025-04-03 02:56:44 -07:00
saltyfish66
e41549c3d6 fix: fix illegal cuda memory access at fused_moe_kernel (#4727) 
Co-authored-by: yuethe <yuethe@tencent.com>
 2025-04-03 00:07:32 -07:00
Jinyan Chen
23c764b18a [Feature] Support DeepEP Low Latency (#4767) 
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: laixinn <xielx@shanghaitech.edu.cn>
Co-authored-by: ch-wan <cwan39@gatech.edu>
 2025-04-01 09:23:25 -07:00