Plan index-skip cache and transfer reduction

Browse Source

Record the phased design for turning NSA index-topk sharing from a compute-only optimization into HiCache and PD-transfer savings. The plan deliberately separates metadata validation from allocation compaction so prefill/decode state-slot mismatches fail before any bandwidth reduction can corrupt cache contents.

Constraint: CP shared-KV cache corruption previously appeared only on cache-hit paths, so state-layer mapping must be explicit before compact transfer or allocation.

Rejected: Compact L1/L2 index buffers first | without state_layer_ids this can silently map different logical layers to the same transfer slot across prefill and decode.

Confidence: high

Scope-risk: narrow

Directive: Do not skip the state_layer_ids phase before compacting NSA index state buffers.

Tested: Documentation placeholder scan and referenced source-path existence check.

Not-tested: Runtime behavior; this commit is documentation only.

...

This commit is contained in:

laoyao0822

2026-06-10 04:28:26 +08:00

parent a32050d1d8

commit 6229c7da60

1 changed files with 1108 additions and 0 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

1108

docs/advanced_features/nsa_prefill_cp_index_skip_cache_communication_plan.md Normal file

File diff suppressed because it is too large Load Diff