Files
sglang/docs/advanced_features
laoyao0822 2317952a01 Document MLA prefix prefetch before Phase 8 implementation
Phase 8 needs a narrow boundary because previous shared-KV phases already changed persistent layout and materialize behavior. The plan records an MLA-only, one-layer-ahead prefix prefetch path that targets chunked-prefill/radix-hit sync materialize overhead without adding index prefetch or bandwidth scheduling yet.

Constraint: Phase 8 must not introduce SGLANG_CP_SHARED_KV_LAYER_PREFETCH_KIND

Constraint: Current request is documentation only; do not modify runtime code

Rejected: Add index K/scale prefetch to the same phase | widens topk correctness and collective-order risk before MLA overlap is validated

Rejected: Persist dense KV across chunks | reintroduces large full-context memory pressure

Confidence: high

Scope-risk: narrow

Directive: Keep Phase 8 v1 MLA-only unless profiling proves index materialize is the next blocker

Tested: git diff --check -- docs/advanced_features/nsa_prefill_cp_phase8_mla_prefix_prefetch_plan.md

Not-tested: Runtime/server execution; documentation-only commit
2026-05-02 07:05:41 +08:00
..
2025-11-16 19:36:33 -08:00