Make CP shared-KV HiCache eviction collective-free and scan-resistant by
removing every per-rank wall-clock input from the eviction/admission decisions
(any such input desyncs the must-be-replicated victim set / batch across the 8
CP ranks and can deadlock the collective-coupled writeback/load-back).
- Replicated logical clock: TreeNode.next_access_time() (a process-global
monotonic counter, mirrors mamba/swa's get_last_access_time) replaces
time.monotonic() as the source of last_access_time at every radix bump site
(radix_cache __init__/match/insert; hiradix CP match/insert/_insert_helper_host;
reset() zeroes it). creation_time/pin_expiry intentionally stay wall-clock.
Because match/insert events are replicated (reqs broadcast from rank 0 over the
replicated tree), last_access_time is now identical on every rank. Unique ints
also give a strict total order (no LRU tie ambiguity). No duration arithmetic
reads last_access_time, so non-CP LRU is unchanged; mamba/swa use their own
TreeNode and are untouched.
- CpReplicatedSLRUStrategy = (is_protected[hit>=2], last_access_time, node.id):
scan-resistant (cold one-shots stay probationary, evicted before reused/
protected prefixes), recency-within-segment ages out cold (not LFU), node.id
is the deterministic total-order tiebreak (heaps never compare TreeNodes).
Overridden for CP so it reaches every get_priority eviction site; the host
write-admission key _cp_host_evict_key is switched from FIFO-by-id to it.
- Co-fixes for the same per-rank-wall-clock class: pin_prefix is forbidden under
CP (its pin_expiry victim-eligibility check is per-rank wall-clock; SLRU
scan-resistance covers the benefit); the affinity head_age_s admission input is
disabled (=0.0, relying on the replicated head_defer_count bound); and a
server_args fail-fast guards CP shared-KV against SGLANG_REQ_WAITING_TIMEOUT>0
(its waiting-queue pruning is per-rank wall-clock).
Tests: test_evict_policy.py adds TestCpReplicatedSLRUStrategy (scan-resistance,
recency, total-order, full ordering) -- 30 passed in the dev-cu13 container;
test_radix_cache_unit.py adds logical-clock determinism/total-order tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
leavelet
8f1e85a992