Phase 8 needs a narrow boundary because previous shared-KV phases already changed persistent layout and materialize behavior. The plan records an MLA-only, one-layer-ahead prefix prefetch path that targets chunked-prefill/radix-hit sync materialize overhead without adding index prefetch or bandwidth scheduling yet.
Constraint: Phase 8 must not introduce SGLANG_CP_SHARED_KV_LAYER_PREFETCH_KIND
Constraint: Current request is documentation only; do not modify runtime code
Rejected: Add index K/scale prefetch to the same phase | widens topk correctness and collective-order risk before MLA overlap is validated
Rejected: Persist dense KV across chunks | reintroduces large full-context memory pressure
Confidence: high
Scope-risk: narrow
Directive: Keep Phase 8 v1 MLA-only unless profiling proves index materialize is the next blocker
Tested: git diff --check -- docs/advanced_features/nsa_prefill_cp_phase8_mla_prefix_prefetch_plan.md
Not-tested: Runtime/server execution; documentation-only commit
Phase 5 needs each current KV page to have exactly one CP compute owner before local KV/index direct writes can be safe. This change teaches in-seq NSA prefill CP to produce page-aligned split metadata under shared-KV mode, threads page size into the metadata builders, and fixes local pair splitting so unequal page-aligned zigzag segments do not corrupt topk inputs.
Constraint: Phase 5 direct-write layout requires page ownership to be expressible at page granularity
Constraint: Short page-unit batches remain on the token-balanced fallback to avoid zero-page segment risk
Rejected: Split local q/weights by half | page-aligned zigzag segments can have unequal token counts
Confidence: medium
Scope-risk: moderate
Directive: Do not enable compute-owner direct writes unless nsa_cp_metadata.page_aligned is true and local loc ownership is verified
Tested: python3 -m py_compile python/sglang/srt/layers/attention/nsa/utils.py python/sglang/srt/layers/attention/nsa/nsa_indexer.py python/sglang/srt/models/deepseek_v2.py python/sglang/srt/models/deepseek_nextn.py test/registered/unit/layers/test_nsa_cp_utils.py
Not-tested: Local pytest collection is blocked in this environment by missing pybase64; container/runtime tests were not rerun during this commit step
Phase 4 is now documented as the page-aligned CP split step, leaving the existing shared KV layout untouched while establishing the page ownership invariant needed for direct writes. Phase 5 is documented as the follow-up compute-owner layout change that can route local MLA KV and NSA index writes into the owning rank's physical pool.
Constraint: Current Phase 2/3 layout is page-interleaved and still relies on compatibility materialize paths.
Rejected: Combine split alignment, allocator changes, and shard-aware attention into one phase | too broad to debug safely.
Confidence: high
Scope-risk: narrow
Directive: Do not start compute-owner KV layout work before preserving the Phase 4 page-aligned split invariant and fallback semantics.
Tested: git diff --cached --check
Not-tested: Runtime server startup; documentation-only commit.
Phase 2 expanded prefill CP persistent KV capacity by sharding KV at rest, but its compatibility runtime can still rebuild full-view KV/index buffers from sharded storage. This documents a narrower Phase 3 target: reuse current chunk KV/index that was already CP all-gathered and reranged, while leaving deeper history shard-aware work to Phase 4.
The new plan separates current-only and mixed current/history cases, defines fallback requirements to the Phase 2 full-view compatibility path, and records validation criteria before any runtime implementation starts.
Constraint: Phase 3 must reduce avoidable duplicate materialization without rewriting NSA topk or sparse attention kernels
Constraint: History KV/index access remains Phase 2 compatibility materialization until Phase 4
Rejected: Start with shard-aware topk/distributed attention | too broad for Phase 3 and harder to validate against the current Phase 2 baseline
Rejected: Remove all shared-KV runtime materialization immediately | history/prefix/chunked-prefill paths still need a compatibility fallback
Confidence: high
Scope-risk: narrow
Directive: Keep Phase 3 scoped to current chunk reuse; put selected-history and shard-aware algorithms in Phase 4 unless the design is explicitly revised
Tested: git diff --check for the two documentation files
Not-tested: Runtime behavior; documentation-only change
Prefill CP previously replicated NSA/MLA persistent KV on every CP rank, so CP8 consumed eight copies of KV memory while exposing only one rank of logical cache capacity. This change splits logical KV locs from per-rank physical storage, shards MLA latent KV and NSA index K/scale by deterministic page ownership, and keeps existing NSA attention kernels working through a full-view runtime materialization layer.
Mooncake PD transfer now sends each prefill CP rank's owned physical pages with explicit logical page positions so non-CP decode can reconstruct full-layout KV. The implementation is guarded by an explicit server flag and startup checks, and the design documentation records the implemented scope, debug environment, and Phase 3 boundary.
Constraint: Phase 2 must preserve existing NSA attention/index kernels via runtime full-view materialization
Constraint: Decode side remains non-CP and receives full KV through Mooncake
Rejected: Shard-aware NSA attention in this change | belongs to Phase 3 because it requires distributed topk/softmax/output contracts
Rejected: Request-contiguous CP ownership | unstable under chunked prefill and tied to attention split mode
Confidence: medium
Scope-risk: broad
Directive: Do not enable round-robin CP shared KV without wiring runtime materialization/PD transfer contracts for that split mode
Directive: Keep SGLANG_DEBUG_CP_SHARED_KV disabled for perf measurements; it intentionally enables CUDA-syncing diagnostics
Tested: Remote py_compile for shared-KV touched Python files in g0034 container
Tested: Remote pytest selected cp_shared/shared_kv/nsa suite: 37 passed, 34 deselected
Not-tested: Full GLM5 multi-node throughput/regression run after final doc update
Not-tested: Phase 3 shard-aware runtime, round-robin CP mode, and non-Mooncake PD backends
- nsa_prefill_cp_all_gather.md: explains why there are two all-gathers
in current NSA prefill CP (per-layer KV/index gather vs tail output
hidden gather), with data flow diagrams and involved file lists
- nsa_prefill_cp_phase1_narrow_output_collection.md: detailed Phase 1
design plan covering batch-level eligibility, fallback conditions,
affected modules, KV transfer impact, risks, and test matrix
laoyao0822