2b.2: B1 collective-free shared-pool evict-to-fit + delete the #5 deadlock

Replaces the 2b.1b shared-L2 capacity-miss SKIP with a deterministic
collective-free evict-to-fit, and deletes the #5 deadlock. This is the gate that
makes --enable-cp-shared-physical-l2-hicache safe in a real run.

evict-to-fit (_reserve_write_cp_shared_l2_evict_to_fit, hiradix): on a shared-L2
reserve capacity miss -- which raises identically on every CP rank over the
replicated free list -- snapshot the plannable shared-L2 host leaves (excluding
the node being backed up) in the replicated SLRU order (_cp_host_evict_key =
Phase-1 logical clock + node.id total-order tiebreak), release the coldest one at
a time (each a replicated free-list mutation via the proven
_evict_cp_host_for_write_admission teardown -> evict_cp_host -> allocator.release)
and retry the pooled reserve after each, stopping at the first fit.
Fragmentation-correct (free coalesces in _return_range). Identical victim
set + order + deterministic reserve => every rank evicts the same victims and
stops at the same point (design Thm 1), so NO collective is needed. The snapshot
is one-level by design (a parent promoted to a leaf mid-loop is not re-pushed;
deep cascades skip-and-retry next tick -- missed identically on all ranks).
Exhaustion -> loud rate-limited skip (transient: pinned/in-flight objects hold
the pool), never a hang.

#5 deadlock DELETED: stripped the synchronize_across_ranks all_reduce machinery
(the `while len(heap) and not all_ranks_done()` ReduceOp.MIN host_evict_done_min
closure + the param) from _evict_host_for_physical_slots -- never reached (no
caller passed True) and the headline CP-deadlock shape. evict_host(num_tokens)
still works (single-arg, equivalent non-sync loop).

Opus adversarial review = SHIP (all 7 findings PASS: determinism [_cp_host_evict_key
purely replicated, pin_expiry dormant under CP], teardown byte-equivalence + no
double-free, no infinite recursion + clean abort-on-partial, snapshot safety [no
ancestor/descendant among host leaves], bounded termination, clean #5 strip,
flag-off path untouched). Tests: new test_eight_ranks_evict_to_fit_stays_identical
(fill->miss->release-in-order->retry -> identical placement, exercises coalescing)
+ 88/88 pool suite (syh-dev-new) + import smoke.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-20 16:03:38 +00:00
co-authored by Claude Opus 4.8
parent 0d87bc576f
commit 07c2b9bc57
2 changed files with 154 additions and 34 deletions
@@ -2779,6 +2779,51 @@ class TestCpSharedL2EightRankReserveDeterminism(unittest.TestCase):
allocs[3].reserve("cp_hicache_node:1", PAYLOAD_TARGET_KV, 1) # rank 3 only
self.assertGreater(len({a.placement_digest() for a in allocs}), 1)
def test_eight_ranks_evict_to_fit_stays_identical(self):
# B1 (2b.2): on a shared-L2 capacity miss every rank runs the SAME
# deterministic evict-to-fit -- release the coldest victims (here the
# ascending-node-id analog of the replicated SLRU order) one at a time,
# retrying the pooled reserve after each, stopping at the first success.
# Identical free list + identical victim order + deterministic reserve =>
# every rank evicts the same victims and stops at the same point (design
# Thm 1), so the hiradix _reserve_write_cp_shared_l2_evict_to_fit needs NO
# collective. Also exercises fragmentation: the win comes only after the
# freed ranges COALESCE into a contiguous run.
allocs = self.make_rank_allocators(pages=8)
for nid, npages in ((0, 3), (1, 3), (2, 2)): # fill 3+3+2 = 8/8 pages
self._reserve_on_all(allocs, nid, (PAYLOAD_TARGET_KV,), npages)
for a in allocs:
a.mark_object_committed(f"cp_hicache_node:{nid}")
self.assertEqual(len({a.placement_digest() for a in allocs}), 1)
# New backup needs 4 contiguous pages -> capacity miss on EVERY rank.
for a in allocs:
with self.assertRaises(ValueError):
a.reserve("cp_hicache_node:9", PAYLOAD_TARGET_KV, 4)
a.abort("cp_hicache_node:9") # SF3: drop any partial reserve
victim_order = [f"cp_hicache_node:{nid}" for nid in (0, 1, 2)]
evicted_counts = []
for a in allocs:
evicted = 0
for victim in victim_order:
a.release(victim)
evicted += 1
try:
a.reserve("cp_hicache_node:9", PAYLOAD_TARGET_KV, 4)
break # contiguous run reopened -> fit
except ValueError:
a.abort("cp_hicache_node:9")
evicted_counts.append(evicted)
# Every rank evicted the SAME victims (release 0 -> free=3<4 no fit; release
# 1 -> [0,3)+[3,6) coalesce to [0,6) -> 4 fits) and reached identical placement.
self.assertEqual(set(evicted_counts), {2})
self.assertEqual(len({a.placement_digest() for a in allocs}), 1)
for a in allocs:
self.assertTrue(a.is_committed("cp_hicache_node:2")) # untouched survivor
self.assertIsNotNone(a.get_range("cp_hicache_node:9", PAYLOAD_TARGET_KV))
if __name__ == "__main__":
unittest.main()