 Even Zhou
|
f4fafacc5d
|
Revert "[feature] Ascend NPU graph support (#8027)" (#9348)
|
2025-08-19 10:11:23 -07:00 |
|
|
zyksir
|
6a9d6ca33c
|
fix unexcepted answer in EAGLE mode (#9252)
|
2025-08-16 17:45:36 -07:00 |
|
|
|
94371dbbd6
|
[feature] Ascend NPU graph support (#8027)
Co-authored-by: ronnie_zheng <zl19940307@163.com>
Co-authored-by: yezhifeng (D) <y00897525@china.huawei.com>
Co-authored-by: anon189Ty <Stari_Falcon@outlook.com>
Co-authored-by: Maksim <makcum888e@mail.ru>
Co-authored-by: ssshinigami <44640852+ssshinigami@users.noreply.github.com>
|
2025-08-16 17:25:17 -07:00 |
|
|
Cheng Wan
|
b87aacb5c5
|
[DP Attention] Refactor: adding some utility functions (#9136)
|
2025-08-13 21:08:06 -07:00 |
|
|
valarLipandHAI
|
53f7874ae6
|
refine aiter_backend for mtp (#7279)
Co-authored-by: HAI <hixiao@gmail.com>
|
2025-08-08 11:06:02 -07:00 |
|
|
Cheng Wan
|
cb099d2095
|
[CUDA Graph] save cuda graph memory by using next_token_logits_buffer (#8579)
|
2025-08-03 03:06:47 -07:00 |
|
|
Cheng Wan
|
7a1f7fc504
|
[Feature] Hybrid EP and TP (#8590)
|
2025-07-31 02:53:25 -07:00 |
|
|
Lianmin Zheng
|
ed2e313eb6
|
Clean up server_args, triton cache manager (#8332)
|
2025-07-25 14:14:51 -07:00 |
|
|
Cheng Wan
|
c0fb25e949
|
DP Enhancement (#8280)
|
2025-07-24 21:36:21 -07:00 |
|
|
J
|
0e7a5b2694
|
fix: prevent crashes due to logit bias dimension mismatch (#7685)
|
2025-07-23 15:30:55 -07:00 |
|
|
Jay Zhou
|
99aefa037e
|
Fix eagle3 cuda graph (#8163)
|
2025-07-20 15:28:06 +08:00 |
|
|
Lianmin ZhengandPranjal Shankhdhar
|
5589b75024
|
Add treemask mode to build_eagle_tree & release sgl-kernel 0.2.3 (#7756)
Co-authored-by: Pranjal Shankhdhar <pranjal.ssh@gmail.com>
|
2025-07-05 12:17:05 -07:00 |
|
|
Cheng Wan
|
6c903611ca
|
Fix incorrect spec_num_draft_tokens in draft_extend (#7757)
|
2025-07-05 02:18:16 -07:00 |
|
|
|
886d344964
|
support llama4 eagle3 (#6985)
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
Co-authored-by: Shenggui Li <somerlee.9@gmail.com>
Co-authored-by: Yingyi Huang <yingyihuang2000@outlook.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
|
2025-06-30 22:34:10 -07:00 |
|
|
Lianmin Zheng
|
071a1f51ae
|
[Minor] clean up multimodal processor and tokenizer manager (#7624)
|
2025-06-29 02:50:14 -07:00 |
|
|
|
ed0a0b692c
|
Perormance: Enable cuda graph for dp idle batch (#7269)
Co-authored-by: austindeng <austindeng@tencent.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: ch-wan <cwan39@gatech.edu>
|
2025-06-23 17:34:13 -07:00 |
|
|
Cheng Wan
|
f8d48fd311
|
Fix dtype for idle input in spec decoding (#7456)
|
2025-06-23 11:23:25 -07:00 |
|
|
Cheng Wan
|
ac5010e0ba
|
Fix CUDA Graph Check under Deepep with DP FFN (#7451)
|
2025-06-22 20:35:58 -07:00 |
|
|
Liangsheng Yin
|
05c9bc8956
|
[minor] simplify the TokenToKVPoolAllocator (#7414)
|
2025-06-22 12:37:18 +08:00 |
|
|
Cheng Wan
|
e879d8b7a8
|
[Feature] Comprehensive Hybrid Parallelism Support (#6389)
|
2025-06-20 14:43:11 -07:00 |
|
|
|
10d60cd41b
|
feat: mtp support dp-attention (#6081)
Co-authored-by: austindeng <austindeng@tencent.com>
Co-authored-by: tianqilin.99 <tianqilin.99@bytedance.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
Co-authored-by: ch-wan <cwan39@gatech.edu>
|
2025-06-17 00:33:28 -07:00 |
|
|
Lianmin Zheng
|
1a9c2c9214
|
Fix AMD speculative decoding (#7252)
|
2025-06-16 17:01:33 -07:00 |
|
|
Lianmin Zheng
|
c64290dcb5
|
Use seq_len_fill_value in the cuda graph runners (#7233)
|
2025-06-16 15:57:07 -07:00 |
|
|
Lianmin Zheng
|
53a525bf33
|
[Eagle] Fix kernel call after updating speculative sampling kernels (#7231)
|
2025-06-16 07:25:59 -07:00 |
|
|
Lianmin Zheng
|
b1286a116a
|
[EAGLE] Refactor code for page size > 1 & more simplifications (#7213)
|
2025-06-16 03:04:29 -07:00 |
|
|
Lianmin Zheng
|
fff10809bf
|
Revert "[EAGLE] Refactor code for page size > 1 & more simplifications" (#7210)
|
2025-06-15 02:48:00 -07:00 |
|
|
Lianmin Zheng
|
5f1ab32717
|
[EAGLE] Refactor code for page size > 1 & more simplifications (#7163)
|
2025-06-14 23:16:23 -07:00 |
|
|
Lianmin Zheng
|
38af4f68a9
|
Fix grammar abort & Minor style fixes (#7204)
|
2025-06-14 22:49:41 -07:00 |
|
|
kyle-pena-kuzco
|
b56de8f943
|
Open AI API hidden states (#6716)
|
2025-06-10 14:37:29 -07:00 |
|
|
Lianmin Zheng
|
6406408a70
|
Clean up server_args.py (#7037)
|
2025-06-10 05:34:29 -07:00 |
|
|
Lianmin Zheng
|
019851d099
|
Fix eagle on AMD (#7051)
|
2025-06-10 05:22:40 -07:00 |
|
|
Lianmin Zheng
|
dc0705a504
|
Simplify prepare_extend_after_decode (#6987)
|
2025-06-09 16:39:21 -07:00 |
|
|
Lianmin Zheng
|
0c1f03a23d
|
Sync cuda graph runners (#6976)
|
2025-06-08 16:12:25 -07:00 |
|
|
Lianmin Zheng
|
60fdad7cf3
|
Sync the changes on cuda graph runners (#6932)
|
2025-06-06 18:23:52 -07:00 |
|
|
Ke Bao
|
a2cb5913a0
|
Add draft extend CUDA graph for flashinfer backend (#6805)
|
2025-06-02 01:51:26 -07:00 |
|
|
Ke Bao
|
7e41290082
|
Add draft extend CUDA graph for Triton backend (#6705)
|
2025-05-29 00:13:07 -07:00 |
|
|
Ke BaoandSehoon Kim
|
631950280a
|
Support EAGLE draft extend CUDA graph (#6606)
Co-authored-by: Sehoon Kim <sehoonkim@berkeley.edu>
|
2025-05-27 02:35:17 -07:00 |
|
|
Ke Baoandmerrymercy
|
6ce0ed073b
|
Apply constraint grammar to EAGLE (#6499)
Co-authored-by: merrymercy <lianminzheng@gmail.com>
|
2025-05-21 17:18:41 -07:00 |
|
|
Lifu Huang
|
3cf1473a09
|
Use monotonic clock for interval measurement (#6211)
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
|
2025-05-17 16:49:18 -07:00 |
|
|
|
2e4babdb0a
|
[Feat] Support FlashMLA backend with MTP and FP8 KV cache (#6109)
Co-authored-by: Yingyi <yingyihuang2000@outlook.com>
Co-authored-by: neiltian <neiltian@tencent.com>
Co-authored-by: lukec <118525388+sleepcoo@users.noreply.github.com>
Co-authored-by: kexueyu <kexueyu@tencent.com>
Co-authored-by: vincentmeng <vincentmeng@tencent.com>
Co-authored-by: pengmeng <pengmeng@tencent.com>
|
2025-05-15 00:48:09 -07:00 |
|
|
Lianmin Zheng
|
e8e18dcdcc
|
Revert "fix some typos" (#6244)
|
2025-05-12 12:53:26 -07:00 |
|
|
applesaucethebunandBrayden Zhong
|
d738ab52f8
|
fix some typos (#6209)
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
|
2025-05-13 01:42:38 +08:00 |
|
|
Lianmin ZhengandSangBin Cho
|
fba8eccd7e
|
Log if cuda graph is used & extend cuda graph capture to cuda-graph-max-bs (#6201)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
|
2025-05-12 00:17:33 -07:00 |
|
|
applesaucethebunandBrayden Zhong
|
2ce8793519
|
Add typo checker in pre-commit (#6179)
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
|
2025-05-11 12:55:00 +08:00 |
|
|
Ying Sheng
|
11383cec3c
|
[PP] Add pipeline parallelism (#5724)
|
2025-04-30 18:18:07 -07:00 |
|
|
Lianmin Zheng
|
21514ff5bd
|
Disable flaky eagle tests (#5753)
|
2025-04-25 15:54:39 -07:00 |
|
|
JieXin Liang
|
97cb762bb6
|
[misc] remove is_cuda_available (#5319)
|
2025-04-20 18:16:51 -07:00 |
|
|
u4lr451
|
211c7b31b8
|
Fix: Incorrect parameters passed to forward_batch_generation (#5506) (#5511)
|
2025-04-17 18:49:59 -07:00 |
|
|
Yineng Zhang
|
4cb53ecd0c
|
fix: log warning when disable cuda graph (#5209)
|
2025-04-09 14:16:13 -07:00 |
|
|
fzyzcjy
|
86a876d883
|
Optimize topk operation in llama4 (#5128)
|
2025-04-09 02:50:22 -07:00 |
|