chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
2,662 Commits 1 Branch 0 Tags
e9c6ce461dee92a5088aef9b7b1c436e18557426
Commit Graph

466 Commits

Author SHA1 Message Date
Xiaoyu Zhang
e9c6ce461d sgl scaled_fp8_quant support output padding (#4861) 2025-04-02 23:53:57 +08:00
Jinyan Chen
23c764b18a [Feature] Support DeepEP Low Latency (#4767) 
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: laixinn <xielx@shanghaitech.edu.cn>
Co-authored-by: ch-wan <cwan39@gatech.edu>
 2025-04-01 09:23:25 -07:00
Yuhong Guo
87fafa0105 Revert PR 4764 & 4813 related to R1 RoPE (#4959) 2025-03-31 20:56:58 -07:00
Yineng Zhang
1c63e79756 use fa3 in sgl-kernel (#4954) 2025-03-31 16:14:49 -07:00
Baizhou Zhang
e62d60fe6d [Fix] avoid stream sync and torch compile in prefill for fa3 backend (#4932) 2025-03-30 13:53:44 -07:00
Lianmin Zheng
b26bc86b36 Support page size > 1 + eagle (#4908) 2025-03-30 00:46:23 -07:00
Baizhou Zhang
20c90be23d [Feature] Support FA3 backend for MLA (#4831) 2025-03-28 18:30:14 -07:00
Qingquan Song
044c315970 Make torch compile configurable for biased_grouped_topk (#4749) 2025-03-28 10:57:52 -07:00
Lianmin Zheng
74e0ac1dbd Clean up import vllm in quantization/__init__.py (#4834) 2025-03-28 10:34:10 -07:00
Qingquan Song
6ffb6bd47a Fix fa3 cuda graph page_size > 1 precision and page_size=1 speed (#4855) 2025-03-28 01:35:59 -07:00
Stefan He
26c0f13126 Support Page Size > 1 for FA3 (#4832) 
Co-authored-by: Qingquan Song <ustcsqq@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-03-27 22:07:14 -07:00
Jiaqi
72031173e4 fix: fix typo of comments in w8a8_fp8.py (#4843) 2025-03-27 21:06:47 -07:00
tarinkk
7f19e083c1 Support (1 <= dp < tp) in the dp attention in DeepEP (#4770) 
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
 2025-03-27 17:09:35 -07:00
strgrb
668ecc6c5b Fix ut mla-test-1-gpu-amd (#4813) 
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
 2025-03-27 08:27:51 -07:00
strgrb
886fcbdd09 Use apply_rope_with_cos_sin_cache_inplace for DeepSeek (#4764) 
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
 2025-03-27 01:45:37 -07:00
Stefan He
1b9175cb23 [FA3 Attn Backend] Remove Unnecessary Device Sync for FA3 (#4745) 
Co-authored-by: Yubo Wang <yubowang2019@gmail.com>
 2025-03-27 00:45:11 -07:00
laixin
ae25d36dc6 [3/3] fix dsv3 awq issue (#4719) 
Co-authored-by: AniZpZ <aniz1905@gmail.com>
 2025-03-26 23:13:43 -07:00
Xiaoyu Zhang
04e3ff6975 Support compressed tensors fp8w8a8 (#4743) 2025-03-26 13:21:25 -07:00
Yi Pan
45fdf1f7f3 Fix shared memory OOM on sm86 GPUs. (#4797) 2025-03-26 10:41:53 -07:00
Thysrael
ced35a0649 fix(typo): fix reply to replay in base_attn_backend.py (#4784) 2025-03-26 00:19:12 -07:00
yuhsaun-t
199bb01d00 Add endpoints to dump selected expert ids (#4435) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-03-24 21:34:19 -07:00
lukec
57eec0bfbc fix FlashMLA cudagraph config (#4691) 
Co-authored-by: yinfan98 <1106310035@qq.com>
 2025-03-24 21:06:58 -07:00
Stefan He
4c584fc632 Fix circular imports in gptq.py and unblock test explorer (#4736) 2025-03-24 18:07:08 -07:00
Stefan He
5d7edc8e55 Support FA3 as Attention backend by using --attention-backend fa3 (#4680) 
Co-authored-by: qsong <qsong@linkedin.com>
Co-authored-by: qingquansong <ustcsqq@gmail.com>
 2025-03-23 23:28:11 -07:00
Kyungmin Lee
2a206b22ed Fix RotaryEmbedding when using Triton backend for EXAONE-3.5-2.4B (#4064) 2025-03-23 17:58:12 -07:00
Mick
11577cedb7 refactor: bug fixes and refactor for vlm (#4661) 2025-03-22 22:48:49 -07:00
fzyzcjy
ca75741e86 Support async in DeepEP (#4610) 
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
 2025-03-22 22:39:56 -07:00
fzyzcjy
c6d549e773 Multiple tiny code cleanups (#4608) 2025-03-22 22:39:11 -07:00
xutizhou
c2bd094d6e Optimize Permute Kernel in DeepEP (#4643) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-03-22 14:30:34 -07:00
Adarsh Shirawalmath
f8f9244a61 [Bug Fix] Add partial rotary factor support for Phi-4 and upgrade to transformers v4.50.0 (#3984) 
Co-authored-by: Chayenne <zhaochen20@outlook.com>
 2025-03-22 14:27:39 -07:00
Yun Dai
8cd4250401 [quantization] fix channelwise conversion with scalar weight scale (#4596) 2025-03-22 00:47:52 -07:00
lukec
4c56e5dbee Set deepgemm to the default value in the hopper architecture. (#4613) 2025-03-20 22:03:00 -07:00
Cheng Wan
7b5fc71972 fix SUPPORT_CUTLASS_BLOCK_FP8 flag (#4640) 2025-03-20 21:45:07 -07:00
JieXin Liang
9e93ef3f8e [fix] fix illegal mem access and clean up triton attention backend (#4571) 2025-03-20 02:01:52 -07:00
lukec
b6944f97a6 Support FlashMLA backend cuda graph (#4514) 
Co-authored-by: yinfan98 <1106310035@qq.com>
Co-authored-by: Hongbosherlock <hongbosherlock@gmail.com>
Co-authored-by: ispobock <ispobaoke@163.com>
 2025-03-19 08:25:34 -07:00
Jinyan Chen
f44db16c8e [Feature] Integrate DeepEP into SGLang (#4232) 
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
Co-authored-by: Xuting Zhou <xutingz@nvidia.com>
 2025-03-19 08:16:31 -07:00
strgrb
f9c53cbb42 Create col-major and tma-aligned x_scale for deep_gemm.gemm_fp8_fp8_bf16_nt (#4515) 
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
 2025-03-19 00:02:43 -07:00
JieXin Liang
c0e9a36c5f Optimize Triton decoding kernel for dynamic workload (#4553) 2025-03-18 21:25:38 -07:00
aoshen524
588865f0e0 [Feature] Support Tensor Parallelism and Weight Slicing for Lora (#4274) 
Co-authored-by: ShenAo1111 <1377693092@qq.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-03-18 20:33:07 -07:00
Cheng Wan
3196999f63 Reduce computation and communication in DP attention (#4521) 2025-03-18 13:41:36 -07:00
James Liu
9e0186f352 [Feature] Support EAGLE 3 (#4247) 2025-03-18 07:35:23 -07:00
Yineng Zhang
c16b33ccac cleanup deps 3/n (#4541) 2025-03-18 00:11:36 -07:00
Xiaoyu Zhang
804d250a0d remove useless backend forward in rotary_embedding (#4500) 2025-03-17 23:54:00 -07:00
Xiaoyu Zhang
dd865befde [Hotfix] solve fp8 w8a8 ci test fail (#4531) 2025-03-17 23:17:04 -07:00
Mick
d373a48c98 fix: second_per_grid_ts should be used to get mrope position (#3682) 2025-03-17 18:12:38 -07:00
Xiaoyu Zhang
9b81f9bd34 sglang quant module remove vllm dependency (#4507) 2025-03-17 15:51:59 -07:00
Ke Bao
3ded4b215d Revert "feat: update grouped_topk to support softmax and sigmoid" (#4505) 2025-03-17 11:30:26 -07:00
Lianmin Zheng
82dec1f70b Remove redundant type conversion (#4513) 2025-03-17 05:57:35 -07:00
yiakwy-xpu-ml-framework-team
5f9b2c62ff [ROCm] fix dtype (#4510) 2025-03-17 05:20:50 -07:00
Lianmin Zheng
5493c3343e Fix data parallel + tensor parallel (#4499) 2025-03-17 05:13:16 -07:00