The decisive level-1 instrumentation for the cross-rank logical-loc
divergence hypothesis. Prior traces all keyed PER-RANK (host pools aren't
comparable across ranks) -- which structurally hid the one invariant that
matters: the visible LOGICAL-loc sequence for a reloaded node must be
bit-identical on every CP rank, or the cross-rank gather reads token j from
its owner rank at a slot that rank filled with a different token.
- visible_locs_hash (cache_controller.load_cp): per-node, per-rank khash of
the visible logical locs, emitted for EVERY node incl. zero-owned so the
analyzer can diff across ranks by node_id. Divergence = root cause.
- victim_set (hiradix _evict_cp_load_back_owner_lanes): per-rank device
load-back eviction victim ids. Device eviction is keyed on wall-clock
last_access_time (per-rank, NOT replicated -- unlike host eviction's
deterministic (priority, node.id)); divergent victims is the trigger.
- release_draw (allocator.alloc_pages_with_owners): fires when a lane's free
bucket is exhausted and the deferred-free release bucket is tapped -- the
use-after-free/aliasing secondary.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
leavelet
25c2d6d606