A CP4 prefill server hung 79s in _plan_cp_load_back_owner_lane_evictions (the [HiCache-load]
slow-scan) and was killed by the detokenizer health-check. Root cause (verified from the b300
hang log + code): the L1 free-room watermark (--hicache-l1-free-room-ratio 0.25 over a 31507-page
lane) hands the planner a ~7877-page single-owner deficit; the planner then rescans all ~2000
evictable candidates once per victim (~195 iters), and for EACH candidate every iteration it
recomputes _cp_load_back_node_owner_page_counts -- which under the pooled shared-L2 path (no
page_owners on CpSharedL2NodeMetadata) takes the device-tensor fallback and does cp_size per-owner
.item() device->host syncs. That is ~195 * 2000 * 4 ~= 1.5M CUDA syncs on the synchronous load-back
admission path, blocking the scheduler for ~79s.
Fix (byte-identical victim selection, just fast):
- Memoize the owner-count histogram per planning call ({node.id: counts}); the counts are invariant
while node.value is fixed (the plan does not mutate values), so node.id is a safe key for the plan's
duration. Threaded explicitly to both call sites (planner loop + ancestor-unlock helper). Turns
O(victims*candidates) recomputes into O(distinct nodes).
- Replace the cp_size per-owner sum().item() loop with one bincount().tolist() device->host sync.
Net: ~79s -> ~1s; the eviction plan (victims, planned_freed) is unchanged. bincount == the per-owner
loop proven over 8000 random vectors incl. the (-1)%cp==cp-1 zero-loc edge. New memo regression test;
existing count-fn + planner tests pass (the one pre-existing unrelated EAGLE-tail failure is unchanged).
(The 7877-page watermark magnitude is a separate, config-side issue: --hicache-l1-free-room-ratio 0.25
reserves ~25% of a ~2M-token lane -- ~30x more than a 64K chunk needs; lower it.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
leavelet
4e9b4d05c0