Files
sglang/docs/advanced_features
laoyao0822 6229c7da60 Plan index-skip cache and transfer reduction
Record the phased design for turning NSA index-topk sharing from a compute-only optimization into HiCache and PD-transfer savings. The plan deliberately separates metadata validation from allocation compaction so prefill/decode state-slot mismatches fail before any bandwidth reduction can corrupt cache contents.

Constraint: CP shared-KV cache corruption previously appeared only on cache-hit paths, so state-layer mapping must be explicit before compact transfer or allocation.

Rejected: Compact L1/L2 index buffers first | without state_layer_ids this can silently map different logical layers to the same transfer slot across prefill and decode.

Confidence: high

Scope-risk: narrow

Directive: Do not skip the state_layer_ids phase before compacting NSA index state buffers.

Tested: Documentation placeholder scan and referenced source-path existence check.

Not-tested: Runtime behavior; this commit is documentation only.
2026-06-10 04:28:26 +08:00
..
2025-11-16 19:36:33 -08:00