[PD] Add different TP sizes support for no-MLA models (#6793)

Co-authored-by: shangmingc <csmthu@gmail.com> Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>
2025-06-25 02:00:22 -07:00
parent f3cbd24541
commit e21aa1df67
3 changed files with 265 additions and 27 deletions
@@ -27,6 +27,8 @@ class KVArgs:
    decode_tp_size: int
    # for pp prefill
    prefill_pp_size: int
+    kv_head_num: int
+    page_size: int


 class KVPoll: