CP HiCache perf: loud one-time warning when PLACEMENT_ASSERT is on in a multi-rank run

The B1 placement-replication assert (SGLANG_CP_HICACHE_PLACEMENT_ASSERT) MIN/MAX-reduces a SHA256
placement_digest across the CP group EVERY scheduler tick, on the scheduler thread, BEFORE the forward
launch (2 blocking gloo collectives + an O(free-ranges+objects) hash per tick). It is debug/bring-up
only. The census (docs_internal/cp_hicache_perf_investigation_2026-06-23.md) shows it ~quadruples the
per-tick collective count vs production; the b300 wire-test script sets it =1, so it is the prime
suspect for "poor perf even L2-only". Warn once so an accidental prod run with it on is visible.
No behavior change when off (cached-bool early-return).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-23 18:11:45 +00:00
parent ea4ce31f95
commit ff1804f5cf

View File

@@ -5619,6 +5619,14 @@ class HiRadixCache(RadixCache):
allocator = getattr(self.cache_controller, "cp_shared_l2_page_allocator", None)
if allocator is None:
return
if not getattr(self, "_placement_assert_warned", False):
self._placement_assert_warned = True
logger.warning(
"[CP-HiCache] SGLANG_CP_HICACHE_PLACEMENT_ASSERT is ON: every scheduler tick MIN/MAX-reduces "
"placement_digest across the CP group (2 blocking gloo collectives on the scheduler thread "
"BEFORE forward) + a SHA256 over the whole free-list. This is a DEBUG/bring-up invariant check "
"-- turn it OFF in production (unset the env) for the per-tick critical-path cost to vanish."
)
local = int(allocator.placement_digest(), 16) % (1 << 61)
lo = torch.tensor(local, dtype=torch.int64, device="cpu")
hi = torch.tensor(local, dtype=torch.int64, device="cpu")