chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Prevent unattached CP HiCache write acks from crashing chunked prefill

Browse Source 
Prepared per-layer CP HiCache backups can enqueue their final write ack before the radix node is attached. Chunked prefill exposed this window when a separate catch-up backup made writing_check drain the ack queue and pop an unattached node id.

Keep ready but unattached prepared acks queued until insert either attaches the prepared backup or rollback removes the orphan ack. Also document the reactive host free-room eviction plan separately from this state-machine fix.

Constraint: CP HiCache prepared backup transfer can complete before radix insertion attaches node state

Rejected: Drop unknown ack ids | would orphan a later successful prepared attach and leak write state

Rejected: Chunked-only guard | the invalid assumption is in the generic CP write ack state machine

Confidence: high

Scope-risk: narrow

Directive: Do not drain CP write acks unless every ack id is registered in pending_host_backups or ongoing_write_through

Tested: Remote red-green test_cp_hicache_metadata.py::TestHiRadixCacheCPBackup::test_writing_check_defers_unattached_prepared_ack

Tested: Remote PYTHONPATH=python python -m pytest -q test/registered/unit/mem_cache/test_cp_hicache_metadata.py test/registered/unit/mem_cache/test_cp_hicache_load_back_owner_lanes.py (116 passed)

Tested: python -m py_compile python/sglang/srt/mem_cache/hiradix_cache.py test/registered/unit/mem_cache/test_cp_hicache_metadata.py

Not-tested: Full chunked-prefill ETE replay after this commit

Co-authored-by: OmX <omx@oh-my-codex.dev>
This commit is contained in:
laoyao0822
2026-06-02 05:17:42 +08:00
parent c2d25ff591
commit 5bd68768d9
4 changed files with 313 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
test/registered/unit/mem_cache/test_cp_hicache_metadata.py
View File

@@ -1778,6 +1778,41 @@ class TestHiRadixCacheCPBackup(CustomTestCase):
self.assertEqual(cache.cache_controller.ack_write_queue, [])
self.assertEqual(evicted, [reservation.metadata])
def test_writing_check_defers_unattached_prepared_ack(self):
cache = HiRadixCache.__new__(HiRadixCache)
cache._uses_cp_hicache = True
cache.tp_world_size = 1
cache.enable_storage = False
cache.pending_host_backups = {}
class ReadyEvent:
def query(self):
return True
def synchronize(self):
pass
attached_node = TreeNode()
attached_node.id = 53
dec_locked = []
cache.ongoing_write_through = {53: attached_node}
cache.dec_node_lock_ref = lambda node: dec_locked.append(node.id)
cache.cache_controller = types.SimpleNamespace(
ack_write_queue=[
HiCacheAck(ReadyEvent(), ReadyEvent(), [54]),
HiCacheAck(ReadyEvent(), ReadyEvent(), [53]),
],
)
cache.writing_check()
self.assertEqual(
[ack.node_ids for ack in cache.cache_controller.ack_write_queue],
[[54], [53]],
)
self.assertEqual(cache.ongoing_write_through, {53: attached_node})
self.assertEqual(dec_locked, [])
def test_write_backup_cp_failfast_on_unplanned_reservation_failure(self):
cache = HiRadixCache.__new__(HiRadixCache)
cache._uses_cp_hicache = True