 cctry
|
cad7878964
|
Gather static input buffers for cuda graph (#13676)
|
2025-11-22 13:01:50 -08:00 |
|
|
Binyao Jiang
|
b29769f3b6
|
Move unnecessary input_addr capture under debug mode flag for speed-up (#13690)
|
2025-11-22 11:42:26 -08:00 |
|
|
Ho-Ren (Jack) Chuang
|
3990b84bd3
|
Refactor MHA & MLA KV caches to support FP4 (#13547)
Signed-off-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
|
2025-11-22 11:13:43 -08:00 |
|
|
|
a92afb00c6
|
[Ascend] support Kimi-K2-Thinking (#12759)
Co-authored-by: ZhengdQin <zhengdqin@gmail.com>
Co-authored-by: richhuan <huan_rz@qq.com>
Co-authored-by: ZhengdQin <46387172+ZhengdQin@users.noreply.github.com>
|
2025-11-22 17:33:07 +08:00 |
|
|
|
90a0133515
|
[Auto Sync] Update http_server.py, io_struct.py, scheduler_... (20251120) (#13679)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Zhuqi Li <zhli@x.ai>
|
2025-11-20 23:17:15 -08:00 |
|
|
|
d754ce973e
|
[Piecewise Cuda Graph] rename, refactor and add more logging (#13675)
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
Co-authored-by: Ke Bao <ISPObaoke@163.com>
Co-authored-by: Oasis-Git <ayw.sirius19@gmail.com>
|
2025-11-21 13:28:38 +08:00 |
|
|
Yuan Luoandluoyuan.luo
|
475962a139
|
[VLM] Support Piecewise CUDA Graph for InternVL (#13640)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
|
2025-11-21 13:21:51 +08:00 |
|
|
|
af6bcadcf7
|
[VLM] Support Piecewise CUDA Graph for Qwen2.5-VL (#13055)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Yuhao Yang <yhyang201@gmail.com>
|
2025-11-20 10:23:44 +08:00 |
|
|
Xinyuan Tong
|
a355794905
|
Expend compatibility check for all quantized MoE models (#13465)
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
|
2025-11-20 01:24:27 +08:00 |
|
|
Chen Haozhe
|
6c2e5fcd91
|
[feat][Ascend][Mindspore]: support model-impl of mindspore (#9234)
|
2025-11-19 09:17:47 +08:00 |
|
|
Zijian Zhang
|
aa8ecbda7a
|
model: support JetVLM (#13289)
|
2025-11-18 12:02:03 +08:00 |
|
|
Carlo Mussolini
|
df56139226
|
Adding user defined hooks support (#13217)
|
2025-11-17 23:07:37 +08:00 |
|
|
lixiaolx
|
d368c7451a
|
(1/n)support context parallel with deepseekv3.2-DSA (#12065)
|
2025-11-16 20:12:25 -08:00 |
|
|
Lifu Huang
|
254f62d879
|
Support spec decoding when LoRA is applied to target model (#12903)
|
2025-11-16 13:20:23 -08:00 |
|
|
Zilin Zhu
|
9509c4ccd3
|
[RL] enable offloading hybrid linear attn model (#13336)
|
2025-11-16 17:12:52 +08:00 |
|
|
kyleliang-nv
|
597d416070
|
[feature] Add layerwise NVTX support (#11870)
|
2025-11-15 19:20:56 -08:00 |
|
|
Even Zhou
|
2aec8b6e1b
|
[Feature] Spec-Overlap supporting DP-ATTN; PD-Disaggregation; npugraph mode (#12443)
|
2025-11-15 21:51:07 +08:00 |
|
|
Yongfei Xu
|
d91b16eb16
|
Opt tp: tp attn support tp reduce scattered input (#10568)
|
2025-11-15 02:08:12 -08:00 |
|
|
Zilin Zhu
|
56fc483073
|
[RL] support only do cpu backup on draft model (#13318)
|
2025-11-15 16:31:57 +08:00 |
|
|
Ho-Ren (Jack) ChuangandYichen Wang
|
6d5e16fb1c
|
feat: Add FP4 (E2M1) KV Cache Support for MHA (#12612)
Signed-off-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
Co-authored-by: Yichen Wang <yichen.wang@bytedance.com>
|
2025-11-14 22:31:35 -08:00 |
|
|
Yuwei An
|
eae59b337e
|
Piecewise Cuda Graph Support for gpt-oss model (#13045)
Signed-off-by: Oasis-Git <ayw.sirius19@gmail.com>
|
2025-11-15 09:28:00 +08:00 |
|
|
|
7a2254b260
|
[Auto Sync] Update backend.py, forward_batch_info.py, piece... (20251113) (#13221)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
|
2025-11-13 18:54:11 -08:00 |
|
|
ykcombat
|
dd192a55f4
|
[Feature] Enable CUDA graph for PD-Multiplexing. (#11595)
|
2025-11-14 03:39:40 +08:00 |
|
|
|
5324f37ab3
|
[Ascend]adapt enable-profile-cuda-graph for NPU (#12617)
Co-authored-by: Leo920320 <1530593013@qq.com>
Co-authored-by: liupengcheng <6531487+leo920320@user.noreply.gitee.com>
|
2025-11-12 19:57:24 +08:00 |
|
|
Xiaoyu Zhang
|
9caca6a45c
|
[PieceWise CUDA Graph] Support awq/gptq model in piecewise cudagraph (#12518)
|
2025-11-11 11:56:15 +08:00 |
|
|
Xuchun Shang
|
ac19ce7efb
|
[PP] put pp assert in model runner (#12934)
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
|
2025-11-10 14:58:19 +08:00 |
|
|
|
ddd1440d0f
|
Refactor KTransformers heterogeneous compute with unified GPU-quantization backend (#12834)
Co-authored-by: Chen Hongtao <56470055+chenht2022@users.noreply.github.com>
Co-authored-by: chenht2022 <cht22@mails.tsinghua.edu.cn>
Co-authored-by: skqliao <skqliao@gmail.com>
Co-authored-by: ovowei <1913953267@qq.com>
|
2025-11-09 21:06:32 -08:00 |
|
|
Liangsheng Yin
|
61c7fe7aed
|
Minor code cleanup / improvement for PREBUILT_EXTEND mode (#12948)
|
2025-11-10 11:59:54 +08:00 |
|
|
Ke Bao
|
db24d34603
|
Support piecewise cuda graph for MLA (#11812)
|
2025-11-10 09:13:48 +08:00 |
|
|
Liangsheng Yinandcctry
|
4f65a64666
|
Refactor / Unify event loop across PD-Disagg, Overlap, DP-Attn cases (#12839)
Co-authored-by: cctry <17473714+cctry@users.noreply.github.com>
|
2025-11-10 00:42:50 +08:00 |
|
|
Zijian Zhang
|
3633f8b0cf
|
Add Jet-Nemotron (#12448)
|
2025-11-09 01:32:47 -08:00 |
|
|
Chen1022
|
c67fce160e
|
Fix missing dp_max_padding argument in set_dp_buffer_len (#12812)
|
2025-11-07 21:27:43 +08:00 |
|
|
hlu1
|
bef37d6de8
|
[Deepseek V3.2] Only skip Indexer logits computation when is_extend_without_speculative (#12816)
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
|
2025-11-07 01:34:04 -08:00 |
|
|
Nicolas Castet
|
2340798353
|
Register allgather/reducescatter buffers with symm memory (#12572)
|
2025-11-04 17:11:36 -08:00 |
|
|
Baizhou Zhang
|
d22d044734
|
Revert "Enable memory saver for hybrid model" (#12648)
|
2025-11-04 16:22:06 -08:00 |
|
|
Junrong Lin
|
173e0f704f
|
Enable memory saver for hybrid model (#11974)
|
2025-11-04 14:55:26 +08:00 |
|
|
|
952fbe47cb
|
fix: fix the bug which leads qwen2_5_vl to crash with mixed_chunk (#11330)
Signed-off-by: PanJason <pyyjason@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Yuan Luo <yuan.luo@hotmail.com>
|
2025-11-03 09:26:03 -08:00 |
|
|
harrisonlimh
|
ffb32a8548
|
Conditionally recapture cuda graph after model weight update from disk (#12060)
|
2025-11-03 05:51:27 -08:00 |
|
|
Yuan Luoandluoyuan.luo
|
819fc59123
|
Add prefix for torch symm mem (#12506)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
|
2025-11-02 11:23:05 -08:00 |
|
|
Ho-Ren (Jack) ChuangandYichen Wang
|
76196b3cbf
|
feat: Add FP4 (E2M1) KV Cache Support with Quantization Utilities for MLA (#10078)
Signed-off-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
Co-authored-by: Yichen Wang <yichen.wang@bytedance.com>
|
2025-11-01 22:24:58 -07:00 |
|
|
Xinyuan Tong
|
0bfe1d145c
|
fa3 & trtllm_mha spec overlap (#11874)
|
2025-10-31 17:38:13 -07:00 |
|
|
Ke Baoandyizhang2077
|
a4bf5c6ad2
|
Support Kimi Linear (#12469)
Co-authored-by: yizhang2077 <1109276519@qq.com>
|
2025-10-31 14:03:35 -07:00 |
|
|
ykcombat
|
41efcaeb45
|
[Feature] PD-Multiplexing Context and Scheduler, lazy import spatial. (#12275)
|
2025-11-01 00:40:01 +08:00 |
|
|
Yuhong Guo
|
ab95d35fcb
|
feat: Add Non-intrusive Tensor Dumping for Model Inference (#10566)
|
2025-10-31 12:04:48 +08:00 |
|
|
Even Zhouandliupeng374
|
ce6b17c0f9
|
[Feature] Support DeepSeek MTP on NPU (#11897)
Co-authored-by: liupeng374 <liupeng374@huawei.com>
|
2025-10-30 21:56:56 +08:00 |
|
|
Makcum888e
|
8e2ac2e628
|
[NPU] fix pp_size>1 (#12195)
|
2025-10-30 11:18:36 +08:00 |
|
|
Yingchun Lai
|
ec92b0cefe
|
EPLB: prefer to use physical experts in the same gpu or node (#10874)
|
2025-10-28 21:01:11 -07:00 |
|
|
Yineng Zhang
|
64cf868eba
|
chore: cleanup quant deps (#12268)
|
2025-10-28 02:03:57 -07:00 |
|
|
Yineng Zhang
|
ea39952797
|
Revert "[Feature] PD-Multiplexing Context and Scheduler." (#12267)
|
2025-10-28 02:00:37 -07:00 |
|
|
Xuchun Shang
|
a1f2dc90e4
|
[Bug fix] [PP] fix wrong dtype for quantified model (#12247)
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
|
2025-10-28 01:27:24 -07:00 |
|