Files
sglang/python
leavelet 1d54b0b7ab CP HiCache trace: reload per-page owner-check (the make-or-break invariant)
Two metadata agents verified that all cache-hit-extend metadata (positions,
page_table, cu_seqlens, topk offset, MoE row-selection) is correct and
prefix-aware. The one reload-specific, unguarded thing is the per-page OWNER
replay: load_cp builds selected_logical_locs = node_device_indices[owned_positions]
and explicitly SKIPS re-validating that those pages are owned by this rank (perf;
the allocator's owner assert is gated behind debug_mode). A per-page owner
permutation (correct bytes, wrong logical page) passes the aggregate host-keyed
value-hash but makes attention read the right bytes for the wrong positions ->
reload-only repeated-token garbage.

Add owner_check (level 1): at each reloaded node, owner_for_logical_pages(
selected_logical_locs // page_size) must all == cp_rank; log bad_owner count +
valid/padded/tail_pad. bad_owner>0 = the per-page owner/padding permutation =
root cause, which the aggregate byte-hash cannot detect.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 04:52:17 +00:00
..
2026-06-10 05:54:43 +08:00