CP HiCache perf: loud one-time warning when PLACEMENT_ASSERT is on in a multi-rank run
The B1 placement-replication assert (SGLANG_CP_HICACHE_PLACEMENT_ASSERT) MIN/MAX-reduces a SHA256 placement_digest across the CP group EVERY scheduler tick, on the scheduler thread, BEFORE the forward launch (2 blocking gloo collectives + an O(free-ranges+objects) hash per tick). It is debug/bring-up only. The census (docs_internal/cp_hicache_perf_investigation_2026-06-23.md) shows it ~quadruples the per-tick collective count vs production; the b300 wire-test script sets it =1, so it is the prime suspect for "poor perf even L2-only". Warn once so an accidental prod run with it on is visible. No behavior change when off (cached-bool early-return). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -5619,6 +5619,14 @@ class HiRadixCache(RadixCache):
|
||||
allocator = getattr(self.cache_controller, "cp_shared_l2_page_allocator", None)
|
||||
if allocator is None:
|
||||
return
|
||||
if not getattr(self, "_placement_assert_warned", False):
|
||||
self._placement_assert_warned = True
|
||||
logger.warning(
|
||||
"[CP-HiCache] SGLANG_CP_HICACHE_PLACEMENT_ASSERT is ON: every scheduler tick MIN/MAX-reduces "
|
||||
"placement_digest across the CP group (2 blocking gloo collectives on the scheduler thread "
|
||||
"BEFORE forward) + a SHA256 over the whole free-list. This is a DEBUG/bring-up invariant check "
|
||||
"-- turn it OFF in production (unset the env) for the per-tick critical-path cost to vanish."
|
||||
)
|
||||
local = int(allocator.placement_digest(), 16) % (1 << 61)
|
||||
lo = torch.tensor(local, dtype=torch.int64, device="cpu")
|
||||
hi = torch.tensor(local, dtype=torch.int64, device="cpu")
|
||||
|
||||
Reference in New Issue
Block a user