Lianmin Zheng
|
de167cf5fa
|
Fix request abortion (#6184)
|
2025-05-10 21:54:46 -07:00 |
|
Lianmin Zheng
|
4319978c73
|
Fix data parallel perf regression (#6183)
|
2025-05-10 19:18:35 -07:00 |
|
huangtingwei
|
d2cb3024f2
|
fix bug that gpu0 occupies more memory when hicache is turned on (#5778)
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
|
2025-05-09 15:36:08 -07:00 |
|
yhyang201
|
cec98f1034
|
[Fix] Incorrect Memory Allocation on CUDA:0 by Non-Zero CUDA Processes in TP/DP (#5745)
|
2025-05-08 17:52:26 -07:00 |
|
fzyzcjy
|
cef91b1ed7
|
[PD] Add control to slow down a server (#5572)
|
2025-05-08 01:03:08 -07:00 |
|
fzyzcjy
|
b6cf3532b5
|
Tiny refactor ModelConfig.from_server_args (#5219)
|
2025-05-08 01:02:43 -07:00 |
|
Liangsheng Yin
|
a3e4e9bf9e
|
Better PD initialization (#5751)
|
2025-05-07 01:12:57 +08:00 |
|
Zhiqiang Xie
|
b26cb1c55a
|
Fix problem of large page size with chunked prefill (#6046)
|
2025-05-06 15:19:47 +08:00 |
|
Zhiqiang Xie
|
f8e460930a
|
Fix prefill OOM error in the case of large page size (#5081)
|
2025-05-05 16:02:55 -07:00 |
|
xm:D
|
3409aaab32
|
Support InternVL3 (#5350)
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
|
2025-05-01 22:38:59 -07:00 |
|
Ying Sheng
|
11383cec3c
|
[PP] Add pipeline parallelism (#5724)
|
2025-04-30 18:18:07 -07:00 |
|
liwenju0
|
8fefdd32c7
|
[Feature] add support kimi vl model (#5383)
Co-authored-by: wenju.li <wenju.li@deepctr.cn>
|
2025-04-29 21:31:19 -07:00 |
|
Chang Su
|
28b26dbf48
|
[Bugfix]: fix missing queue_time_start for requests from grammar_queue (#5696)
|
2025-04-29 17:31:44 -07:00 |
|
Ke Bao
|
dd408ee481
|
Auto set draft model path for MTP (#5793)
|
2025-04-29 16:25:40 -07:00 |
|
Lianmin Zheng
|
3029889cb4
|
Turn on overlap scheduler for multimodal models (#5771)
|
2025-04-27 23:45:09 -07:00 |
|
Trevor Morris
|
84810da4ae
|
Add Cutlass MLA attention backend (#5390)
|
2025-04-27 20:58:53 -07:00 |
|
Liangsheng Yin
|
40d9b8acce
|
Improve overlap scheduling (#5788)
|
2025-04-28 11:19:16 +08:00 |
|
Yi Zhang
|
1f963d7f64
|
Bugfix for minicpmo vision test (#5760)
|
2025-04-26 23:18:02 +08:00 |
|
Mick
|
feda9b11b3
|
fix: fix one more bug from merging mm_inputs (#5718)
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: XinyuanTong <115166877+JustinTong0323@users.noreply.github.com>
|
2025-04-25 17:28:33 -07:00 |
|
IAN
|
11e27d0926
|
[PD]: Support Muti Prefill in one node (#5704)
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
|
2025-04-26 00:30:47 +08:00 |
|
Liangsheng Yin
|
c55550cbf0
|
[PD] Better logs (#5715)
|
2025-04-25 17:25:45 +08:00 |
|
Mick
|
c998d04b46
|
vlm: enable radix cache for qwen-vl models (#5349)
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
|
2025-04-23 20:35:05 -07:00 |
|
Cheng Wan
|
711efe7814
|
Integrating PD disaggregation with DP attention and DeepEP (#5435)
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
|
2025-04-23 01:46:01 -07:00 |
|
Byron Hsu
|
bf98d2e377
|
[PD] Support prefill overlap + Ensure no race condition (#5609)
|
2025-04-21 12:12:56 -07:00 |
|
Byron Hsu
|
e65b9f21e3
|
[PD] Support decode overlap schedule (#5608)
|
2025-04-21 12:06:16 -07:00 |
|
Zhiqiang Xie
|
70645f4d7d
|
upstream hicache fixes (#5570)
|
2025-04-20 23:08:30 -07:00 |
|
Lianmin Zheng
|
eef9433b46
|
Fix flush cache (#5590)
|
2025-04-20 22:56:40 -07:00 |
|
fzyzcjy
|
1195182040
|
Tiny add Engine.flush_cache API (#5241)
|
2025-04-20 18:15:03 -07:00 |
|
Sundara Raman Ramachandran
|
f08154193c
|
Perform Batch Tokenization. (#5141)
|
2025-04-20 18:10:37 -07:00 |
|
fzyzcjy
|
5fc4b6004e
|
Add sanity check for max_running_requests (#5016)
|
2025-04-20 17:56:49 -07:00 |
|
fzyzcjy
|
475e2e378a
|
[PD] Fix server crash when using batch requests (#5531)
|
2025-04-20 16:02:23 -07:00 |
|
fzyzcjy
|
fa2f677e18
|
Fix torch memory saver not enabled in DP scenario (#5560)
|
2025-04-20 14:20:52 -07:00 |
|
Xiaoyu Zhang
|
d58e354472
|
simplify the control logic for using shared experts fusion (#5504)
|
2025-04-19 13:17:35 -07:00 |
|
fzyzcjy
|
f6a71139a8
|
Make profiler output file names consistent (#5548)
|
2025-04-18 22:57:11 -07:00 |
|
fzyzcjy
|
53dcf38876
|
Introduce moe_dense_tp_size to fix dense layer errors in DeepSeek V3 + 4x8xH100 (#4836)
|
2025-04-17 21:38:26 -07:00 |
|
Baizhou Zhang
|
6fb29ffd9e
|
Deprecate enable-flashinfer-mla and enable-flashmla (#5480)
|
2025-04-17 01:43:33 -07:00 |
|
Baizhou Zhang
|
4fb05583ef
|
Deprecate disable-mla (#5481)
|
2025-04-17 01:43:14 -07:00 |
|
Lianmin Zheng
|
177320a582
|
Clean up imports (#5467)
|
2025-04-16 15:26:49 -07:00 |
|
Cheng Wan
|
6aca583420
|
Fix several minor issues in PD disaggregation (#5444)
|
2025-04-15 23:04:41 -07:00 |
|
Baizhou Zhang
|
a42736bbb8
|
Support MHA with chunked prefix cache for DeepSeek chunked prefill (#5113)
|
2025-04-15 22:01:22 -07:00 |
|
ybyang
|
dd83e7e9c3
|
[Bug fix] need record start time in pd mode (#5425)
|
2025-04-16 10:11:16 +08:00 |
|
shangmingc
|
ffde65a094
|
[PD] Fix dynamic port support and MLA buffer for Mooncake (#5415)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: ybyang <ybyang7@iflytek.com>
|
2025-04-15 19:29:31 +08:00 |
|
Byron Hsu
|
a9499885e9
|
[PD] Add transfer backend abstraction (#5328)
|
2025-04-14 01:39:39 +08:00 |
|
Liangsheng Yin
|
f765579046
|
Fix typo: infight -> inflight (#5357)
|
2025-04-14 01:25:30 +08:00 |
|
tianlian yi
|
bc92107b03
|
Support server based rollout in Verlengine (#4848)
Co-authored-by: Jin Pan <jpan236@wisc.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Jinn <47354855+jhinpan@users.noreply.github.com>
|
2025-04-12 10:07:52 -07:00 |
|
Mick
|
34ef6c8135
|
[VLM] Adopt fast image processor by default (#5065)
|
2025-04-11 21:46:58 -07:00 |
|
Mick
|
e53a0b3d5b
|
[fix] fix mrope positions not picked up (#5265)
|
2025-04-11 01:29:45 -07:00 |
|
Cheng Wan
|
038bc5d521
|
Support --enable-llama4-multimodal (#5254)
|
2025-04-11 01:24:14 -07:00 |
|
Ke Bao
|
1078396f47
|
Update deps for mllama4 (#5215)
|
2025-04-10 09:12:44 -07:00 |
|
Teng Ma
|
4c31ae9f6d
|
[PD] Support KV transfer with mooncake (#4880)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
Co-authored-by: shangmingc <csmthu@gmail.com>
|
2025-04-10 14:23:23 +08:00 |
|