Files
sglang/python
leavelet ff1804f5cf CP HiCache perf: loud one-time warning when PLACEMENT_ASSERT is on in a multi-rank run
The B1 placement-replication assert (SGLANG_CP_HICACHE_PLACEMENT_ASSERT) MIN/MAX-reduces a SHA256
placement_digest across the CP group EVERY scheduler tick, on the scheduler thread, BEFORE the forward
launch (2 blocking gloo collectives + an O(free-ranges+objects) hash per tick). It is debug/bring-up
only. The census (docs_internal/cp_hicache_perf_investigation_2026-06-23.md) shows it ~quadruples the
per-tick collective count vs production; the b300 wire-test script sets it =1, so it is the prime
suspect for "poor perf even L2-only". Warn once so an accidental prod run with it on is visible.
No behavior change when off (cached-bool early-return).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 18:11:45 +00:00
..
2026-06-10 05:54:43 +08:00