Reduce CP HiCache capacity synchronization to owner-lane logic

CP shared KV and HiCache now use owner-lane metadata as the
authoritative capacity view for host write admission and GPU load-back
planning. This removes the debug scalar capacity env and keeps CP load-back
from relying on a rank-wide scalar collective when per-owner availability is
already known. The load-back planner also accounts for evicting child leaves
that unlock ancestor device residency, which fixes small lane deficits despite
large aggregate evictable capacity.

The commit also adds gated CPU timing logs for CP shared-KV MLA/index
prefetch and a CUDA microbenchmark for comparing dense all-reduce with
owner-packed all-gather layouts. The timing logs are intentionally behind the
existing MLA prefetch log env and should not be enabled for throughput
measurements.

Constraint: CP shared KV owner lanes require target/draft capacity decisions to preserve page_owners rather than total-token scalars
Constraint: CUDA collective benchmarks must run on target GPU hosts, not locally
Rejected: Keep SGLANG_CP_HICACHE_CAPACITY_DEBUG observer env | owner-lane admission now replaces that scalar debug path
Rejected: Add a silent scalar-allreduce fallback | unexpected owner-lane mismatch should fail fast or log loudly
Confidence: medium
Scope-risk: moderate
Directive: Do not reintroduce CP capacity collectives on the scheduler hot path without proving the owner-lane metadata is insufficient
Directive: Disable SGLANG_CP_SHARED_KV_LOG_MLA_PREFETCH for end-to-end performance runs; it is diagnostic and high-volume
Tested: git diff --check
Tested: python -m py_compile on changed runtime/test/benchmark Python files
Tested: remote pytest -q test/registered/unit/mem_cache/test_cp_hicache_load_back_owner_lanes.py test/registered/unit/mem_cache/test_cp_hicache_metadata.py (81 passed, 5 warnings)
Not-tested: CUDA benchmark benchmark/hicache/bench_cp_shared_kv_prefetch_collective.py
Not-tested: full GLM5 E2E throughput after this commit
This commit is contained in:
laoyao0822
2026-05-28 08:31:49 +08:00
parent ff33446787
commit 25f2147677
8 changed files with 996 additions and 201 deletions

View File

@@ -185,12 +185,13 @@ waits for D2H write of layer `i`.
- Planner skips device-valid nodes, pending backup nodes, host-protected nodes,
pinned nodes, malformed metadata, and non-host-backed leaves.
- Current eviction behavior is unchanged.
- **Online debug option added:** `SGLANG_CP_HICACHE_CAPACITY_DEBUG=1`.
- When enabled, write admission computes the deterministic pre-reserve plan and
compares its eviction-needed decision with the current collective
`reserve_slots_max` result.
- Logs use `[HiCache-capacity-debug] write_admission_compare ...`.
- This is observer-only and does not remove or add collectives.
- **Write admission is now owner-lane authoritative.**
- The old observer/debug scalar planner (`current_compatible_need` plus
`SGLANG_CP_HICACHE_CAPACITY_DEBUG`) has been removed from the hot path.
- Write admission uses the per-owner target/draft deficit vector directly,
evicts deterministic host victims only for lanes with deficit, and fails
fast if a subsequent local reservation failure contradicts the vector view.
- No CP capacity all-reduce is used for write admission.
### P1: Observer-only ledger

View File

@@ -226,17 +226,12 @@ all_reduce per node victim in a tight host-eviction loop
all_reduce every scheduler tick when no completion prefix advanced
```
Current correctness note: CP host reservation now synchronizes
`required_host_slots` with `all_reduce(MAX)` before the host-eviction retry.
This forces every rank into the same reserve/evict/retry branch and avoids
collective mismatches when one rank is host-full and another rank reserves
successfully. It is intentionally a coarse slow-path collective, not a
per-layer collective, but it can become a performance cost when host pressure
is frequent because every reservation failure pays at least one rank-wide MAX
sync and retry failures pay a second one. Treat this as a correctness guard to
be amortized later with batched reservation epochs, deterministic host
watermarks, or less frequent proactive host eviction; do not move it into the
per-layer data path.
Current correctness note: CP host reservation no longer uses the old
`required_host_slots` scalar `all_reduce(MAX)`. The owner-lane vector computed
from `CpHiCacheNodeMetadata.page_owners` is now the authoritative admission
view for target and draft host capacity. A reservation failure after this
vector predicts no deficit is treated as an invariant violation rather than
falling back to another collective.
## Per-Layer Backup Data Plane
@@ -466,10 +461,11 @@ paths:
2. **Pending split behavior.** The chosen first pass is defer/requeue the
request that would split a node with pending backup. Do not split the
in-flight backup op, including for future `bs > 1`.
3. **Ack batching threshold.** Current `writing_check()` can all-reduce on
every progress poll. The first per-layer implementation should keep one
final logical ack per node and check it at final visibility time, not per
layer; batching threshold for final commit remains a later performance pass.
3. **Ack batching threshold.** `writing_check()` no longer enters the TP
all-reduce while a write is ongoing but no final ack exists. The remaining
`MIN` is the final host-visibility barrier for ack entries that do exist:
it should not run per layer, and removing it requires a different global
visibility protocol rather than a local-only commit.
4. **Failure policy under reserve mismatch.** Capacity pressure should skip
backup; malformed target/draft metadata should fail fast. If a local rank
fails reservation after deterministic host eviction while another succeeds,