Keep CP HiCache draft KV invisible until joint readiness

CP HiCache now treats draft KV as a strict target-owned payload through pending write visibility, host eviction, and state-buffer registration. Host metadata created before async D2H ack is no longer request-visible, so match_prefix cannot select an in-flight host node. Draft host eviction now fails before target cleanup when draft metadata is missing, and prefill/decode share one helper for draft NSA state buffers so shared-KV mode cannot silently skip mismatched draft state.

Constraint: CP shared KV + HiCache + EAGLE/MTP must not expose target-only host hits or skipped draft state as valid cache hits

Rejected: Rely on event-loop ordering and lock_ref to hide in-flight writes | match_prefix does not consult lock_ref and can observe host_len/cp_hicache directly

Rejected: Keep draft state mismatch as debug-only skip | it can poison speculative acceptance while looking like a successful cache hit

Confidence: high

Scope-risk: moderate

Directive: Do not reintroduce silent draft/target fallback in CP shared-KV HiCache paths; malformed strong-sync metadata should fail fast

Tested: python -m py_compile targeted modified files

Tested: remote g0034 container pytest test/registered/unit/mem_cache/test_cp_hicache_metadata.py test/registered/unit/managers/test_hicache_controller_cp.py test/registered/unit/disaggregation/test_cp_shared_kv_transfer_mapping.py -q (91 passed)

Not-tested: Full CP shared KV + HiCache + EAGLE/MTP ETE server run after this commit
This commit is contained in:
laoyao0822
2026-05-27 05:46:34 +08:00
parent 71c4f66968
commit 367dff06f3
9 changed files with 448 additions and 42 deletions

View File

@@ -35,6 +35,14 @@ eviction, or visibility decisions.
zero-owned rank handling and frees the replay allocation on failure.
- Phase 3 implemented: `HiRadixCache.reset()` clears target and draft host pools
as one logical reset.
- Phase 5 implemented: CP host-backed predicate now treats in-flight
write-through nodes as pending, not request-visible host hits; duplicate
write-through launch is suppressed while a node is in
`ongoing_write_through`.
- Phase 6 implemented: controller tests now cover target+draft host eviction,
zero-owned draft metadata, and draft-before-target load readiness; CP draft
state-buffer registration is centralized and fails fast on target/draft state
mismatch when `SGLANG_CP_DRAFT_SHARED_KV=1`.
- Adjacent controller invariant guard restored for CPU/fake-test tensors only:
page-shaped host/device indices are validated in unit tests without
reintroducing CUDA tensor truth-value checks or hot-path host sync.
@@ -574,6 +582,14 @@ the explicit-ready path before per-layer backup. Pending write is a valid
lifecycle state, so it may return “not ready”; malformed metadata must still
raise.
Implemented decision: use the explicit-ready path. `_node_backuped()` still
validates CP/draft metadata first, but returns `False` while the node id is in
`ongoing_write_through`. This keeps in-flight D2H metadata out of request-visible
host-hit admission until `writing_check()` observes the joint target+draft ack
and removes the node from the pending table. `_inc_hit_count()` treats pending
write-through as “already launched” and does not start a second write for the
same node.
### Phase 6: Keep internal transfer split, but expose one logical operation
Files:
@@ -615,6 +631,23 @@ Add controller tests:
an error. It should not silently skip draft state buffers because that creates
a target/draft cache-hit mismatch.
Implemented status:
- `evict_cp_host()` fails before freeing target host slots if draft HiCache is
attached but metadata lacks `draft_host_indices`; otherwise target and draft
host slots are freed as one logical operation.
- `_write_cp()` zero-owned ranks keep explicit empty `draft_host_indices`, so
zero-owned target metadata is still draft-valid.
- `start_loading()` is test-protected to submit draft H2D before completing the
corresponding target layer readiness event.
- `append_cp_draft_state_buffers()` is the single helper used by prefill and
decode queue setup. It appends draft NSA state buffers to the target state
stream, preserves legacy skip behavior when draft shared KV is disabled, and
raises on target/draft state mismatch when draft shared KV is enabled.
- Code-read confirmation: PD transfer page indices remain target-driven;
draft KV buffers are extra buffers behind the same page-index stream, not an
independent draft page-index stream.
### Phase 7: Remote validation
Run targeted tests in the remote container because this repo's runtime deps and