chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

[PD] Fix dynamic port support and MLA buffer for Mooncake (#5415)

Browse Source 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: ybyang <ybyang7@iflytek.com>
This commit is contained in:
shangmingc
2025-04-15 19:29:31 +08:00
committed by GitHub
parent 471650dee0
commit ffde65a094
6 changed files with 171 additions and 180 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
python/sglang/srt/disaggregation/decode.py
View File

@@ -128,8 +128,11 @@ class DecodePreallocQueue:
metadata_buffer[0].nbytes for metadata_buffer in self.metadata_buffers
]
kv_args.ib_device = "mock-ib-device"
kv_args.gpu_id = self.scheduler.gpu_id
kv_manager_class = get_kv_class(self.transfer_backend, KVClassType.MANAGER)
kv_manager = kv_manager_class(kv_args, DisaggregationMode.DECODE)
kv_manager = kv_manager_class(
kv_args, DisaggregationMode.DECODE, self.scheduler.server_args
)
return kv_manager
def add(self, req: Req) -> None: