L2 prereqs for CP8DP2 + L3: scope HiCache consensus to the CP group + reset pooled allocator on flush

Two latent CP shared-KV L2 correctness fixes, landed BEFORE L3 (not in an L3
commit). Surfaced by a per-CP8-instance (CP8DP2EP16) scoping review.

PREREQ-1 (CP-group scoping). The B1 commit/evict consensus collectives
(writing_check ReduceOp.MIN, placement_digest MIN/MAX + its tp_world_size<=1
entry guard, drain_storage_control_queues, the evict/prefetch MINs, the flush
barrier) AND cache_controller.prefetch_tp_group all derive from
self.tp_group/self.tp_world_size, which was params.tp_cache_group. tp_cache_group
equals the CP group ONLY at dp_size=1 (enable_dp_attention False -> tp_cpu_group,
and attn_cp_size==tp_size -> _ATTN_CP==_TP) -- the sole reason B1 works today.
Under CP8DP2 (DP attention, attn_tp_size=1) tp_cache_group is the size-1 attn-TP
group, so every `tp_world_size>1` collective silently no-ops per rank and the
placement assert self-disables -> divergent placement -> shared-slab corruption.
Fix: for CP hicache (cp_size>1) scope self.tp_group to the CP cpu group
(get_attention_cp_group().cpu_group -- already used for the slab-handle
broadcast) + self.tp_world_size to its size. A no-op handle change at dp_size=1
(same group object); the intended fix at CP8DP2. The single init-point change
propagates to every consensus collective + un-gates prefetch_tp_group + re-enables
the placement assert.

PREREQ-2 (flush reset). HiRadixCache.reset() cleared the radix tree + host pool
but never reset CpSharedL2PageAllocator -> stale free list/ranges/committed after
flush_cache (leak; the shared pool was never reclaimed). Added
CpSharedL2PageAllocator.reset() (rebuild the per-slab free list all-free, drop
ranges + committed, restore the freshly-built placement_digest) called from
cache_controller.reset() after the ack queues are cleared. Safe: flush_cache is
idle-gated (no in-flight backup/reserve). This is also L3's clear hookpoint.

Validation: new test_reset_restores_freshly_constructed_all_free_state + 89/89
pool suite (torch-2.11 container) + import smoke. PREREQ-1 is a no-op at dp_size=1
(live no-regression confirmed on the next prefill restart); CP8DP2 correctness is
by construction (CP-group membership verified) pending a 2-machine run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-20 21:25:34 +00:00
co-authored by Claude Opus 4.8
parent 9549b268d7
commit 9368c33b23
4 changed files with 70 additions and 1 deletions
@@ -1019,6 +1019,28 @@ class TestCpSharedL2PageAllocator(unittest.TestCase):
all(src == 0 and group == "fake-cpu-group" for src, group, _ in broadcasts)
)
def test_reset_restores_freshly_constructed_all_free_state(self):
# PREREQ-2: flush_cache must reclaim the pooled L2. reset() rebuilds the
# free list all-free + drops ranges/committed, restoring the freshly-built
# placement_digest exactly (rank-uniform at the idle flush point).
allocator = self.make_allocator(pages=8)
fresh_digest = allocator.placement_digest()
allocator.reserve("n0", PAYLOAD_TARGET_KV, 3)
allocator.reserve("n1", PAYLOAD_TARGET_KV, 2)
allocator.mark_object_committed("n0")
self.assertLess(allocator.free_pages(PAYLOAD_TARGET_KV), 8)
self.assertNotEqual(allocator.placement_digest(), fresh_digest)
allocator.reset()
self.assertEqual(allocator.free_pages(PAYLOAD_TARGET_KV), 8)
self.assertEqual(allocator.free_pages(PAYLOAD_DRAFT_KV), 8)
self.assertFalse(allocator.is_committed("n0"))
self.assertEqual(allocator.object_ranges("n0"), {})
self.assertEqual(allocator.placement_digest(), fresh_digest)
# usable after reset
self.assertIsNotNone(allocator.reserve("n2", PAYLOAD_TARGET_KV, 4))
def test_release_and_abort_return_capacity_once_without_double_free(self):
allocator = self.make_allocator(pages=5)
allocator.reserve("release-me", PAYLOAD_TARGET_KV, 2)