Files
sglang/python/sglang/srt/configs
laoyao0822 d21952b903 Reduce inactive NSA index-cache transfer safely
Centralize the IndexCache skip formula and thread the resulting active logical index layers into NSA KV pools. HiCache now skips only the indexer H2D/D2H payload for inactive target layers while preserving per-layer MLA KV transfer, keeping allocation shape unchanged for this phase.

Constraint: P0-P2 must not compact device or host allocation yet; prefill/decode state transfer still has no logical layer-id metadata.

Rejected: Recompute the skip formula separately in mem_cache | formula drift would corrupt cache or waste transfers when offset/pattern settings change.

Rejected: Skip whole-layer HiCache load/backup | MLA KV remains required for every attention layer.

Confidence: medium

Scope-risk: moderate

Directive: Before enabling compact state buffers or compact allocation, add layer-id metadata validation to PD transfer.

Tested: Local py_compile for touched files; remote pytest in g0034 container: test_nsa_index_layers.py and TestNSAIndexerPageIndices, 20 passed.

Not-tested: ETE replay/GSM8K with --nsa-index-topk-freq 4; PD state-transfer compaction remains unimplemented.
2026-06-10 04:28:26 +08:00
..
2025-09-30 12:18:39 -07:00
2026-02-16 00:18:31 +08:00
2026-02-16 00:18:31 +08:00
2025-11-18 12:02:03 +08:00
2026-01-27 10:57:00 +08:00
2025-11-05 15:08:15 -08:00
2025-12-04 14:31:26 -08:00