chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[Feature] LMCache Connector Integration (#9741)

Browse Source 
Signed-off-by: Oasis-Git <ayw.sirius19@gmail.com>
Signed-off-by: YuhanLiu11 <yliu738@wisc.edu>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
This commit is contained in:
Yuwei An
2025-09-06 20:14:55 -07:00
committed by GitHub
parent cb3918a091
commit 9a7ced4e4d
7 changed files with 478 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
python/sglang/srt/mem_cache/memory_pool.py
View File

@@ -369,7 +369,6 @@ class MHATokenToKVPool(KVCache):
# same applies to get_value_buffer and get_kv_buffer
if self.layer_transfer_counter is not None:
self.layer_transfer_counter.wait_until(layer_id - self.start_layer)
return self._get_key_buffer(layer_id)
def _get_value_buffer(self, layer_id: int):