Files
sglang/python
laoyao0822 c739ea667d Keep CP HiCache backup state replicated on radix nodes
Rank-local CP HiCache async write queues can drain at different times across ranks. Radix structural decisions that consult those local queues can diverge, so backup-pending state now lives on the replicated radix node and uses a separate backup lock ref from request locks. Commit and rollback clear the replicated marker, and split/prune/hit-count probes use the marker rather than local pending dictionaries.

Constraint: CP ranks must make identical radix structural decisions even when local async write acknowledgements drain at different times

Rejected: Drain write acknowledgements before every split/prune decision | still rank-local and adds hot-path synchronization

Rejected: Treat backup lock_ref as a normal request lock | hides the distinction between request protection and backup-pending protection

Confidence: high

Scope-risk: moderate

Directive: Do not derive CP HiCache split/prune/write-through decisions from pending_host_backups or ongoing_write_through without a replicated node marker

Tested: local py_compile python/sglang/srt/mem_cache/hiradix_cache.py python/sglang/srt/mem_cache/radix_cache.py

Tested: local git diff --check

Tested: remote cjy-glm5-new pytest replicated marker test plus targeted CP HiCache metadata subset, 10 passed

Not-tested: full test_cp_hicache_metadata.py because current branch has unrelated fixture drift around enable_cp_l3/cp_shared_l2_page_allocator/running-count setup

Co-authored-by: OmX <omx@oh-my-codex.dev>
2026-06-26 08:17:41 +08:00
..
2026-06-10 05:54:43 +08:00