Qiaolin Yu
|
8c0cfca87d
|
Feat: support cuda graph for LoRA (#4115)
Co-authored-by: Beichen Ma <mabeichen12@gmail.com>
|
2025-04-28 23:30:44 -07:00 |
|
Lianmin Zheng
|
849c83a0c0
|
[CI] test chunked prefill more (#5798)
|
2025-04-28 10:57:17 -07:00 |
|
Trevor Morris
|
84810da4ae
|
Add Cutlass MLA attention backend (#5390)
|
2025-04-27 20:58:53 -07:00 |
|
Lianmin Zheng
|
621e96bf9b
|
[CI] Fix ci tests (#5769)
|
2025-04-27 07:18:10 -07:00 |
|
Kebe
|
7e944246c3
|
Add memory_saver check (#4986)
Signed-off-by: Kebe <mail@kebe7jun.com>
|
2025-04-26 20:20:05 -07:00 |
|
vzed
|
df2cf583ce
|
we fix the non existent access of decrypted_config_file (#5685)
|
2025-04-26 18:32:37 -07:00 |
|
Mick
|
c998d04b46
|
vlm: enable radix cache for qwen-vl models (#5349)
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
|
2025-04-23 20:35:05 -07:00 |
|
Yineng Zhang
|
04f2abcb34
|
fix: gemma 3 not use softcap (#5622)
|
2025-04-22 01:16:08 -07:00 |
|
Lianmin Zheng
|
1343200299
|
Clean up mem settings (#5610)
|
2025-04-21 17:19:00 -07:00 |
|
Liangsheng Yin
|
e69a219074
|
Enhance GPU memory settings (#5604)
|
2025-04-21 15:15:00 -07:00 |
|
Byron Hsu
|
bf98d2e377
|
[PD] Support prefill overlap + Ensure no race condition (#5609)
|
2025-04-21 12:12:56 -07:00 |
|
Byron Hsu
|
e65b9f21e3
|
[PD] Support decode overlap schedule (#5608)
|
2025-04-21 12:06:16 -07:00 |
|
Trevor Morris
|
4dce1cc608
|
[PD] Add NIXL transfer backend (#5477)
|
2025-04-22 01:36:12 +08:00 |
|
fzyzcjy
|
dc86f25a57
|
Tiny remove duplicated code (#5021)
|
2025-04-21 00:47:32 -07:00 |
|
Zhiqiang Xie
|
70645f4d7d
|
upstream hicache fixes (#5570)
|
2025-04-20 23:08:30 -07:00 |
|
Qingquan Song
|
188f0955fa
|
Add Speculative Decoding Eagle3 topk > 1 (#5318)
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
Co-authored-by: Yubo Wang <yubowang2019@gmail.com>
|
2025-04-20 22:58:28 -07:00 |
|
Sundara Raman Ramachandran
|
f08154193c
|
Perform Batch Tokenization. (#5141)
|
2025-04-20 18:10:37 -07:00 |
|
Yi Zhou
|
fac17acf08
|
add function call parser for DeepSeek V3 (#5224)
|
2025-04-20 17:38:08 -07:00 |
|
JieXin Liang
|
bca832c7c6
|
[Fix] fix outlines and xgrammar (#4947)
|
2025-04-20 13:31:25 -07:00 |
|
Xiaoyu Zhang
|
d58e354472
|
simplify the control logic for using shared experts fusion (#5504)
|
2025-04-19 13:17:35 -07:00 |
|
shangmingc
|
dca90f1db8
|
[PD] Remove the requirement of config file for mooncake backend (#5460)
|
2025-04-19 19:31:00 +08:00 |
|
fzyzcjy
|
53dcf38876
|
Introduce moe_dense_tp_size to fix dense layer errors in DeepSeek V3 + 4x8xH100 (#4836)
|
2025-04-17 21:38:26 -07:00 |
|
Baizhou Zhang
|
6fb29ffd9e
|
Deprecate enable-flashinfer-mla and enable-flashmla (#5480)
|
2025-04-17 01:43:33 -07:00 |
|
Baizhou Zhang
|
4fb05583ef
|
Deprecate disable-mla (#5481)
|
2025-04-17 01:43:14 -07:00 |
|
Lianmin Zheng
|
177320a582
|
Clean up imports (#5467)
|
2025-04-16 15:26:49 -07:00 |
|
Baizhou Zhang
|
a42736bbb8
|
Support MHA with chunked prefix cache for DeepSeek chunked prefill (#5113)
|
2025-04-15 22:01:22 -07:00 |
|
Byron Hsu
|
a9499885e9
|
[PD] Add transfer backend abstraction (#5328)
|
2025-04-14 01:39:39 +08:00 |
|
Mick
|
34ef6c8135
|
[VLM] Adopt fast image processor by default (#5065)
|
2025-04-11 21:46:58 -07:00 |
|
Cheng Wan
|
038bc5d521
|
Support --enable-llama4-multimodal (#5254)
|
2025-04-11 01:24:14 -07:00 |
|
Jinyan Chen
|
bc3f6db2dd
|
[Fix] DeepEP Compatibility with Low Latency (#5068)
Co-authored-by: ch-wan <cwan39@gatech.edu>
|
2025-04-08 20:31:31 -07:00 |
|
Trevor Morris
|
11d760d56a
|
FP4 weight loading and inference (2/2) (#3972)
|
2025-04-08 17:26:21 -07:00 |
|
Baizhou Zhang
|
efbae697b3
|
[Revision] Replace enable_flashinfer_mla argument with attention_backend (#5052)
|
2025-04-05 01:23:02 -07:00 |
|
Cheng Wan
|
4c54f44202
|
[deepep] fix: shared experts are not initialized when shared experts fusion is enabled (#5072)
|
2025-04-04 15:08:30 -07:00 |
|
Xiaoyu Zhang
|
924ca7c92c
|
Add DeepSeek V3/R1 shared experts fusion (#4918)
|
2025-04-04 01:59:29 -07:00 |
|
AniZpZ
|
d95269f9b3
|
[2/3] fix dsv3 awq issue (#4625)
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: laixinn <xielx@shanghaitech.edu.cn>
|
2025-04-03 17:36:39 -07:00 |
|
Lianmin Zheng
|
74885a848b
|
Revert "Replace enable_flashinfer_mla argument with attention_backend" (#5048)
|
2025-04-03 13:30:56 -07:00 |
|
fzyzcjy
|
8e10fec9a8
|
Small refactor DeepEPMode to clean up code a bit (#4992)
|
2025-04-03 02:56:44 -07:00 |
|
Baizhou Zhang
|
e8999b13b7
|
Replace enable_flashinfer_mla argument with attention_backend (#5005)
|
2025-04-03 02:53:58 -07:00 |
|
Jinyan Chen
|
23c764b18a
|
[Feature] Support DeepEP Low Latency (#4767)
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: laixinn <xielx@shanghaitech.edu.cn>
Co-authored-by: ch-wan <cwan39@gatech.edu>
|
2025-04-01 09:23:25 -07:00 |
|
Lianmin Zheng
|
4ede6770cd
|
Fix retract for page size > 1 (#4914)
|
2025-03-30 02:57:15 -07:00 |
|
Lianmin Zheng
|
b26bc86b36
|
Support page size > 1 + eagle (#4908)
|
2025-03-30 00:46:23 -07:00 |
|
fzyzcjy
|
cf29fe9e78
|
Fix Engine error when enabling DP attention (#4648)
|
2025-03-27 22:17:30 -07:00 |
|
Vincent
|
e2e2ab70e0
|
IPv6 support (#3949)
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
|
2025-03-27 21:42:13 -07:00 |
|
tarinkk
|
7f19e083c1
|
Support (1 <= dp < tp) in the dp attention in DeepEP (#4770)
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
|
2025-03-27 17:09:35 -07:00 |
|
Xiaoyu Zhang
|
04e3ff6975
|
Support compressed tensors fp8w8a8 (#4743)
|
2025-03-26 13:21:25 -07:00 |
|
Stefan He
|
5d7edc8e55
|
Support FA3 as Attention backend by using --attention-backend fa3 (#4680)
Co-authored-by: qsong <qsong@linkedin.com>
Co-authored-by: qingquansong <ustcsqq@gmail.com>
|
2025-03-23 23:28:11 -07:00 |
|
Byron Hsu
|
c7c7dbebbe
|
[PD] Release initial code (#4654)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Ying1123 <sqy1415@gmail.com>
Co-authored-by: merrymercy <lianminzheng@gmail.com>
Co-authored-by: makro
Co-authored-by: dhou-xai
|
2025-03-21 14:47:47 -07:00 |
|
lukec
|
b6944f97a6
|
Support FlashMLA backend cuda graph (#4514)
Co-authored-by: yinfan98 <1106310035@qq.com>
Co-authored-by: Hongbosherlock <hongbosherlock@gmail.com>
Co-authored-by: ispobock <ispobaoke@163.com>
|
2025-03-19 08:25:34 -07:00 |
|
Jinyan Chen
|
f44db16c8e
|
[Feature] Integrate DeepEP into SGLang (#4232)
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
Co-authored-by: Xuting Zhou <xutingz@nvidia.com>
|
2025-03-19 08:16:31 -07:00 |
|
James Liu
|
9e0186f352
|
[Feature] Support EAGLE 3 (#4247)
|
2025-03-18 07:35:23 -07:00 |
|