 lambert0312
|
a086a11305
|
Use sgl-kernel sgl_per_token_group_quant_int8 (#4971)
|
2025-04-26 20:19:42 -07:00 |
|
|
saltyfish66
|
133ded039a
|
perf: update H20 fused_moe_triton kernel config to get higher throughput during prefilling (#5716)
|
2025-04-26 18:15:07 -07:00 |
|
|
Xiaoyu Zhang
|
18ce468d56
|
update triton 3.2.0 h200 fused moe triton config and add warning about triton fused_moe_kernel performance degradation due to different Triton versions. (#5740)
|
2025-04-25 16:24:59 -07:00 |
|
|
saltyfish66andyuethe
|
2aa3f5e2d0
|
[feature] Add H20 fp8_w8a8 FusedMoE config for --n-share-experts-fusion=16 (#5641)
Co-authored-by: yuethe <yuethe@tencent.com>
|
2025-04-22 09:33:13 -07:00 |
|
|
fzyzcjy
|
463d4b7400
|
Fix DeepEP cannot run on latest master (#5567)
|
2025-04-20 14:19:42 -07:00 |
|
|
Xiaoyu Zhang
|
d58e354472
|
simplify the control logic for using shared experts fusion (#5504)
|
2025-04-19 13:17:35 -07:00 |
|
|
fzyzcjy
|
1e0806f30b
|
Fix DeepGEMM masked cannot be run on groups not being multiple or 4 (#5340)
|
2025-04-18 22:38:07 -07:00 |
|
|
Xiaoyu Zhang
|
bed05878f6
|
fix kimi vl running bug after rebase main (#5461)
|
2025-04-18 00:17:34 -07:00 |
|
|
Lianmin Zheng
|
177320a582
|
Clean up imports (#5467)
|
2025-04-16 15:26:49 -07:00 |
|
|
Ximingwang-09andximing.wxm
|
2dd6489468
|
Add H20 dtype fp8_w8a8 shared experts fused MoE kernel tuning configs for DeepSeek V3/R1 (#5291)
Co-authored-by: ximing.wxm <ximing.wxm@antgroup.com>
|
2025-04-14 18:40:31 -07:00 |
|
|
lambert0312
|
61e7c4dd21
|
Add A800 shared experts fused MoE kernel tuning configs for DeepSeek V3/R1 (#5368)
|
2025-04-14 18:39:44 -07:00 |
|
|
Xiaoyu ZhangandYineng Zhang
|
38076dea84
|
apply fused moe gate in ds v3/r1 (#5371)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-04-14 16:24:26 -07:00 |
|
|
Xiaoyu Zhang
|
3e4794aad8
|
refine fused_moe tuning docs (#5294)
|
2025-04-12 10:01:13 -07:00 |
|
|
Xiaoyu Zhang
|
690ec20587
|
Delete python/sglang/srt/layers/moe/fused_moe_triton/configs/E=257,N=… (#5321)
|
2025-04-12 10:00:03 -07:00 |
|
|
Xiaoyu Zhang
|
60bcbf2a35
|
remove moe_align_block_size torch.zeros in small batch/expert mode (#5298)
|
2025-04-11 12:13:55 -07:00 |
|
|
Xiaoyu Zhang
|
f730362ee2
|
reduce moe_align_block_size_kernel small batch mode overhead (#5086)
|
2025-04-09 17:59:35 -07:00 |
|
|
Zhaoyang Hao
|
456b008bd8
|
Add H20 dtype fp8_w8a8 fused MoE kernel tuning configs for DeepSeek V3/R1 (#5196)
|
2025-04-09 11:54:36 -07:00 |
|
|
|
4065248214
|
Support Llama4 fp8 inference (#5194)
Co-authored-by: laixinn <xielx@shanghaitech.edu.cn>
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: zhyncs <me@zhyncs.com>
|
2025-04-09 20:14:34 +08:00 |
|
|
Cheng Wan
|
76c48a0913
|
[DeepEP] fix: import buffer error (#5179)
|
2025-04-08 22:12:14 -07:00 |
|
|
Jinyan Chenandch-wan
|
bc3f6db2dd
|
[Fix] DeepEP Compatibility with Low Latency (#5068)
Co-authored-by: ch-wan <cwan39@gatech.edu>
|
2025-04-08 20:31:31 -07:00 |
|
|
|
5a144a8ab9
|
Fix run time error in ROCm platform (#5147)
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: root <root@dell300x-pla-t10-17.pla.dcgpu>
|
2025-04-07 22:49:40 -07:00 |
|
|
Hubert Lu
|
afb752bcbe
|
[AMD] Fix missing per_token_group_quant_fp8 for ROCm (#5140)
|
2025-04-07 22:38:25 -07:00 |
|
|
|
f04c80dc42
|
Add Llama4 support (#5092)
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
Co-authored-by: ispobock <ispobaoke@163.com>
|
2025-04-07 00:29:36 -07:00 |
|
|
Xiaoyu Zhang
|
924ca7c92c
|
Add DeepSeek V3/R1 shared experts fusion (#4918)
|
2025-04-04 01:59:29 -07:00 |
|
|
fzyzcjy
|
6ff9c6a5e7
|
Cleanup unused resources after DeepEP operation (#4996)
|
2025-04-04 00:36:04 -07:00 |
|
|
fzyzcjy
|
77e929a1a2
|
Support async DeepEP by splitting into two stages (#4995)
|
2025-04-04 00:32:27 -07:00 |
|
|
fzyzcjy
|
febe21ce03
|
Small refactor DeepEPDispatcher into subclasses (#4994)
|
2025-04-04 00:24:18 -07:00 |
|
|
Tommy Yang
|
31035dda44
|
Add H20 fused MoE kernel tuning configs for DeepSeek V3/R1 (#5057)
|
2025-04-03 22:14:28 -07:00 |
|
|
|
d95269f9b3
|
[2/3] fix dsv3 awq issue (#4625)
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: laixinn <xielx@shanghaitech.edu.cn>
|
2025-04-03 17:36:39 -07:00 |
|
|
fzyzcjy
|
8e10fec9a8
|
Small refactor DeepEPMode to clean up code a bit (#4992)
|
2025-04-03 02:56:44 -07:00 |
|
|
saltyfish66andyuethe
|
e41549c3d6
|
fix: fix illegal cuda memory access at fused_moe_kernel (#4727)
Co-authored-by: yuethe <yuethe@tencent.com>
|
2025-04-03 00:07:32 -07:00 |
|
|
|
23c764b18a
|
[Feature] Support DeepEP Low Latency (#4767)
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: laixinn <xielx@shanghaitech.edu.cn>
Co-authored-by: ch-wan <cwan39@gatech.edu>
|
2025-04-01 09:23:25 -07:00 |
|
|
Qingquan Song
|
044c315970
|
Make torch compile configurable for biased_grouped_topk (#4749)
|
2025-03-28 10:57:52 -07:00 |
|
|
Lianmin Zheng
|
74e0ac1dbd
|
Clean up import vllm in quantization/__init__.py (#4834)
|
2025-03-28 10:34:10 -07:00 |
|
|
Xiaoyu Zhang
|
04e3ff6975
|
Support compressed tensors fp8w8a8 (#4743)
|
2025-03-26 13:21:25 -07:00 |
|
|
yuhsaun-tandCheng Wan
|
199bb01d00
|
Add endpoints to dump selected expert ids (#4435)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
|
2025-03-24 21:34:19 -07:00 |
|
|
fzyzcjyandCheng Wan
|
ca75741e86
|
Support async in DeepEP (#4610)
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
|
2025-03-22 22:39:56 -07:00 |
|
|
fzyzcjy
|
c6d549e773
|
Multiple tiny code cleanups (#4608)
|
2025-03-22 22:39:11 -07:00 |
|
|
xutizhouandCheng Wan
|
c2bd094d6e
|
Optimize Permute Kernel in DeepEP (#4643)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
|
2025-03-22 14:30:34 -07:00 |
|
|
|
f44db16c8e
|
[Feature] Integrate DeepEP into SGLang (#4232)
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
Co-authored-by: Xuting Zhou <xutingz@nvidia.com>
|
2025-03-19 08:16:31 -07:00 |
|
|
Ke Bao
|
3ded4b215d
|
Revert "feat: update grouped_topk to support softmax and sigmoid" (#4505)
|
2025-03-17 11:30:26 -07:00 |
|
|
Wenbo Yang
|
75b656488a
|
Support serving DeepSeek-R1-Channel-INT8 with 32 L40S. (#4418)
|
2025-03-17 00:03:43 -07:00 |
|
|
Mick
|
0f52fb55ec
|
config: Update fused moe config (#4493)
|
2025-03-16 23:51:58 -07:00 |
|
|
Mick
|
8ec2ce0726
|
perf: update fused moe config (#4459)
|
2025-03-15 21:23:57 -07:00 |
|
|
Yineng Zhang
|
ad1ae7f7cd
|
use topk_softmax with sgl-kernel (#4439)
|
2025-03-14 15:59:06 -07:00 |
|
|
Yineng Zhangandsleepcoo
|
977d7cd26a
|
cleanup deps 1/n (#4400)
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
|
2025-03-14 00:00:33 -07:00 |
|
|
|
c6d7f8d370
|
Add some fused elementwise kernels for grok-1 (#4398)
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
|
2025-03-13 13:39:10 -07:00 |
|
|
Cheng WanandAbatom
|
2f6bacee03
|
[moe] fix: correct the cache size in the last chunk (#3679)
Co-authored-by: Abatom <abzhonghua@gmail.com>
|
2025-03-12 22:22:13 -07:00 |
|
|
Stefan HeandStefan He
|
e0917e6bd0
|
Remove vllm ops scaled fp8 quant and accelerate per token quant by 20-28% (#4215)
Co-authored-by: Stefan He <bhe@linkedin.com>
|
2025-03-12 00:08:03 -07:00 |
|
|
lambert0312
|
7140ba3573
|
Add A800 tuning configs for DeepSeek R1/V3 channel-wise INT8 (#4323)
|
2025-03-11 18:25:56 -07:00 |
|