CP shared KV and HiCache now use owner-lane metadata as the
authoritative capacity view for host write admission and GPU load-back
planning. This removes the debug scalar capacity env and keeps CP load-back
from relying on a rank-wide scalar collective when per-owner availability is
already known. The load-back planner also accounts for evicting child leaves
that unlock ancestor device residency, which fixes small lane deficits despite
large aggregate evictable capacity.
The commit also adds gated CPU timing logs for CP shared-KV MLA/index
prefetch and a CUDA microbenchmark for comparing dense all-reduce with
owner-packed all-gather layouts. The timing logs are intentionally behind the
existing MLA prefetch log env and should not be enabled for throughput
measurements.
Constraint: CP shared KV owner lanes require target/draft capacity decisions to preserve page_owners rather than total-token scalars
Constraint: CUDA collective benchmarks must run on target GPU hosts, not locally
Rejected: Keep SGLANG_CP_HICACHE_CAPACITY_DEBUG observer env | owner-lane admission now replaces that scalar debug path
Rejected: Add a silent scalar-allreduce fallback | unexpected owner-lane mismatch should fail fast or log loudly
Confidence: medium
Scope-risk: moderate
Directive: Do not reintroduce CP capacity collectives on the scheduler hot path without proving the owner-lane metadata is insufficient
Directive: Disable SGLANG_CP_SHARED_KV_LOG_MLA_PREFETCH for end-to-end performance runs; it is diagnostic and high-volume
Tested: git diff --check
Tested: python -m py_compile on changed runtime/test/benchmark Python files
Tested: remote pytest -q test/registered/unit/mem_cache/test_cp_hicache_load_back_owner_lanes.py test/registered/unit/mem_cache/test_cp_hicache_metadata.py (81 passed, 5 warnings)
Not-tested: CUDA benchmark benchmark/hicache/bench_cp_shared_kv_prefetch_collective.py
Not-tested: full GLM5 E2E throughput after this commit
The per-layer backup path did not explain end-to-end performance by inspection alone, so add a focused benchmark that compares all-layer tail backup against per-layer forward-overlap backup under the same page_first_direct layout. The benchmark reports total latency, over-forward-only latency, submit overhead, and winner so short-extend fallback decisions are based on measured cost rather than request-size intuition.
Constraint: CUDA benchmark execution is remote-only; this local commit only records the benchmark driver.
Rejected: Adding more online logs for this question | the user asked for benchmark evidence and logs would perturb scheduler behavior.
Confidence: high
Scope-risk: narrow
Directive: Do not infer production policy from per_extra alone; per-layer CPU submission can be hidden inside the forward timing interval, so total latency is the safer comparison metric.
Tested: Local py_compile benchmark/hicache/bench_cp_hicache_backup_overlap.py.
Tested: Remote benchmark logs generated under /mnt/beegfs/cjy/cp_hicache_backup_overlap_bench_20260527_*.log.
Not-tested: Full SGLang test suite; benchmark is standalone and CUDA execution was remote-only.
laoyao0822
leavelet