 Shangming Cai
|
d98a4913ea
|
[PD] Refactor parallel sizes and add pp support for mooncake (#8571)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-08-04 20:18:11 -07:00 |
|
|
Shangming Cai
|
016fd25127
|
[PD] Use batch transfer for rdma transport and add notes for mnnvl usage (#8595)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-07-31 21:29:34 +08:00 |
|
|
Shangming Cai
|
55ecdc0a8e
|
Update CODEOWNERS (#8562)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-07-30 01:05:57 -07:00 |
|
|
Shangming Caiandybyang
|
2fd5c7049f
|
[PD] Fix abort_request for PD disaggregation (#8352)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: ybyang <10629930+whybeyoung@users.noreply.github.com>
|
2025-07-27 21:48:27 -07:00 |
|
|
Shangming Cai
|
22e00eeb4a
|
[Bugfix] Prevent PD server crash from invalid grammar (#8062)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-07-28 00:17:51 +08:00 |
|
|
Shangming Cai
|
70e37b97bf
|
chore: upgrade mooncake 0.3.5 (#8341)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-07-25 01:17:26 -07:00 |
|
|
Shangming Cai
|
1403ea5694
|
[PD] Support non-MLA models PD different TP with DP attention (#7931)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-07-18 22:00:49 -07:00 |
|
|
Shangming Cai
|
64c5907e12
|
[PD] Add guidance for prefill bootstrap timeout (#7846)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-07-08 21:00:34 -07:00 |
|
|
Shangming Cai
|
d379bda4fa
|
[Bugfix] Fix two batch overlap with auto DeepEP Dispatch (#7853)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-07-08 02:49:32 -07:00 |
|
|
Shangming Cai
|
2ff572e28c
|
[CI][Router] Fix bench_one_batch_server for pd router test (#7731)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-07-02 23:18:24 -07:00 |
|
|
Shangming Cai
|
5c2142579a
|
[PD] Raise error for incompatible mooncake version and some minor fixes (#7527)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-25 18:55:24 -07:00 |
|
|
Shangming Cai
|
a07f8ae4b7
|
[CI] Upgrade mooncake to v0.3.4.post2 to fix potential slice failed bug (#7522)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-25 01:49:22 -07:00 |
|
|
Shangming Cai
|
7b9a174a7a
|
[PD][Spec] Fix hidden state transfer for spec decode (#7516)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-25 00:42:07 -07:00 |
|
|
Shangming Cai
|
d6dddc19ff
|
[CI] Upgrade mooncake to 0.3.4.post1 to fix 8 gpu tests (#7472)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-24 02:10:50 +08:00 |
|
|
Shangming Cai
|
187b85b7f3
|
[PD] Optimize custom mem pool usage and bump mooncake version (#7393)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-20 09:50:39 -07:00 |
|
|
Shangming Cai
|
f88e70853e
|
[Bugfix][PD] Set conclude state before clear when failure happens (#7362)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-19 11:26:53 -07:00 |
|
|
shangmingc
|
c26d7349d3
|
[PD] Add custom memory pool option to support Mooncake PD with NVLink (#7264)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-17 17:21:37 -07:00 |
|
|
shangmingc
|
ceaa85c9e6
|
[PD] Support get local ip from NIC for PD disaggregation (#7237)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-17 17:19:26 -07:00 |
|
|
shangmingc
|
132dad874d
|
[PD] Optimize transfer queue forward logic for dummy rank (#6922)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-06 18:26:14 -07:00 |
|
|
shangmingc
|
dd1012fcbe
|
[PD] Fix potential perf spike caused by tracker gc and optimize doc (#6764)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-05 10:56:02 -07:00 |
|
|
shangmingc
|
6cb00c6398
|
[PD] Optimize time out logic and add env var doc for mooncake (#6761)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-30 00:45:02 -07:00 |
|
|
shangmingc
|
1dc6864f17
|
[PD] Support completion endpoint (#6729)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-29 16:26:18 +08:00 |
|
|
shangmingc
|
d63e76f735
|
[CI] Fix setup of disaggregation with different tp (#6706)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-28 11:17:27 -07:00 |
|
|
shangmingc
|
e9fd11c0d1
|
[Bugfix] Fix ChatCompletion endpoint of mini_lb when stream is set (#6703)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-28 21:33:36 +08:00 |
|
|
shangmingc
|
c7588d593e
|
[Bugfix] Fix slice operation when chunk size mismatch (#6697)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-28 21:15:00 +08:00 |
|
|
shangmingc
|
b1c8d4e9f3
|
[PD] Abort unbootstrapped prefill requests through timeout (#6685)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-28 00:40:54 -07:00 |
|
|
shangmingc
|
c25231c679
|
[CI] Fix flaky pp single node test (#6689)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-28 00:40:26 -07:00 |
|
|
shangmingc
|
fba03b29e3
|
[Bugfix] Fix missing abort finish reason for PD with ChatCompletion (#6693)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-28 00:39:46 -07:00 |
|
|
shangmingc
|
3ce94f71f9
|
[PD] Handle P/D failure and reconnect without affecting other instances (#6263)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-26 19:21:01 -07:00 |
|
|
shangmingc
|
58f10679e1
|
Fix missing http status import for PD failure handler (#6520)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-22 15:23:54 +08:00 |
|
|
shangmingc
|
f1c896007a
|
[PD] Add support for different TP sizes per DP rank (#5922)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-12 13:55:42 -07:00 |
|
|
shangmingc
|
3ee40ff919
|
[CI] Re-enable pd disaggregation test (#6231)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-12 10:09:12 -07:00 |
|
|
shangmingc
|
0f334945c6
|
[CI] Fix PD mooncake dependency error (#6212)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-12 10:08:49 -07:00 |
|
|
shangmingc
|
31d1f6e7f4
|
[PD] Add simple unit test for disaggregation feature (#5654)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-11 13:35:27 +08:00 |
|
|
shangmingc
|
56f6589ecb
|
[PD] Optimize disaggregation ib device help info (#5781)
|
2025-05-05 13:47:37 +08:00 |
|
|
shangmingc
|
50eda8398e
|
[PD] Add kvargs table and thread pool for kvcache sender of mooncake (#5738)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-04-25 18:15:01 +08:00 |
|
|
shangmingc
|
e0673969b9
|
[PD] Add support for dp attention with mooncake (#5530)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-04-23 17:20:27 +08:00 |
|
|
shangmingc
|
f29a718f63
|
[PD] Fix generate endpoint of min_lb for PD (#5598)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-04-21 21:39:18 +08:00 |
|
|
shangmingc
|
dca90f1db8
|
[PD] Remove the requirement of config file for mooncake backend (#5460)
|
2025-04-19 19:31:00 +08:00 |
|
|
shangmingc
|
f1b3b75fc6
|
[PD] Remove unused bootstrap param and fix port table type (#5423)
|
2025-04-15 21:21:20 +08:00 |
|
|
shangmingcandybyang
|
ffde65a094
|
[PD] Fix dynamic port support and MLA buffer for Mooncake (#5415)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: ybyang <ybyang7@iflytek.com>
|
2025-04-15 19:29:31 +08:00 |
|