Files
sglang/python
laoyao0822 9549b268d7 Avoid allocating indexer state for shared NSA layers
Target-model skip-topk layers reuse the previous active layer's top-k indices and should not run local indexer modules. Centralize the layer-needs-indexer decision, skip constructing indexers on shared target layers, and skip their checkpoint tensors during load while keeping nextn/draft layers conservative for state safety.

Constraint: index skip should reduce GPU memory in both prefill and decode without changing top-k propagation semantics
Constraint: nextn/draft layers report shared top-k behavior but still need local indexer state safety
Rejected: Loader-only filtering | parameters are already allocated during model construction
Rejected: Dummy indexer modules for skipped layers | preserves most of the memory cost this change removes
Confidence: high
Scope-risk: moderate
Directive: Do not reintroduce indexer execution on skip_topk target layers without proving prev_topk propagation and weight residency semantics
Tested: remote g0034 cjy-glm5-new PYTHONPATH=python python -m pytest -q test/registered/unit/speculative/test_spec_utils.py test/registered/unit/configs/test_nsa_index_layers.py test/registered/unit/models/test_deepseek_index_skip_weight_loading.py -> 19 passed
Tested: remote g0034 cjy-glm5-new py_compile for modified runtime files
Not-tested: full GLM5 model restart memory delta measurement
2026-06-21 05:24:05 +08:00
..
2026-06-10 05:54:43 +08:00