2b.3 fixes: restore two l2_pooling-adopt gaps surfaced by live validation
The flag-on 2b.3 bring-up (--enable-cp-shared-physical-l2-hicache, real GLM-5.1 PD traffic) surfaced two partial-adopt gaps from the piecemeal 2b.0/2b.1b ports from l2_pooling — a caller brought in without its callee/declaration. Neither was caught by the pool unit tests or import smoke (they don't exercise the device-allocator load path or the flag-on capacity-snapshot path); only live flag-on traffic hit them. 1. environ.py: declare SGLANG_CP_HICACHE_VERIFY_SNAPSHOT (EnvBool, default False). hiradix _cp_host_capacity_snapshot (:1477) reads it to gate a fail-loud running-count drift verifier (_cp_walk_capacity_counts vs _cp_counts), but the env field was never declared -> AttributeError crash on the first flag-on scheduler tick. The verifier IS fully implemented (it's the count-machinery analog of the placement_digest assert) -- keep it, default-off (expensive full-walk; on in bring-up/CI). 2. allocator.py: restore alloc_pages_for_shared_l2_load (thin wrapper over the adopted alloc_pages_with_owners) on the CP shared paged allocator. 2b.0 ported the callee but dropped this wrapper that cache_controller.load_cp_shared_l2 calls -> RuntimeError on every L2->L1 cache-hit reload until restored. Validated live: prefill boots with the flag, shared-L2 slab constructs over hugetlbfs on all 8 ranks, write-through + load-back + cache hits + coherent output all work, placement_digest assert green (no divergence), no crash. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -393,6 +393,10 @@ class Envs:
|
||||
# is replicated (Theorem 1). When on, MIN/MAX-reduce placement_digest across the
|
||||
# CP group each tick and fail loud on divergence. On in CI/bring-up, off in prod.
|
||||
SGLANG_CP_HICACHE_PLACEMENT_ASSERT = EnvBool(False)
|
||||
# CP HiCache debug drift-detector: when on, _cp_host_capacity_snapshot walks the
|
||||
# full radix tree and asserts the running per-rank counts match (fail loud on
|
||||
# drift). Expensive full-walk per snapshot -> default off; on in bring-up/CI.
|
||||
SGLANG_CP_HICACHE_VERIFY_SNAPSHOT = EnvBool(False)
|
||||
|
||||
# Mooncake KV Transfer
|
||||
SGLANG_MOONCAKE_CUSTOM_MEM_POOL = EnvStr(None)
|
||||
|
||||
@@ -1045,6 +1045,20 @@ class CPSharedPagedTokenToKVPoolAllocator(PagedTokenToKVPoolAllocator):
|
||||
|
||||
return out_indices
|
||||
|
||||
def alloc_pages_for_shared_l2_load(
|
||||
self, current_page_owners: List[int]
|
||||
) -> Optional[torch.Tensor]:
|
||||
"""Allocate fresh L1 pages for shared-physical L2 load-back.
|
||||
|
||||
``current_page_owners`` is the load-time owner plan recomputed
|
||||
positionally ((page-1) % cp_size), NOT the write-time ``page_owners``
|
||||
metadata used by legacy CP HiCache replay. Thin wrapper over
|
||||
``alloc_pages_with_owners``; restored here because the 2b.0 adopt
|
||||
ported the callee but dropped this wrapper that
|
||||
``cache_controller.load_cp_shared_l2`` calls.
|
||||
"""
|
||||
return self.alloc_pages_with_owners(current_page_owners)
|
||||
|
||||
def alloc_pages_with_owners(
|
||||
self,
|
||||
page_compute_owners: List[int],
|
||||
|
||||
Reference in New Issue
Block a user