Files
sglang/python
laoyao0822 df3cc9abf8 Reuse draft MTP indexer topk only when model config allows
EAGLE V2 draft MTP can avoid recomputing NSA/DSA indexer topk across internal draft iterations when the model declares that those indices are shareable. The port follows the upstream narrow contract: store per-forward topk on ForwardBatch, enable reuse only for topk=1, and clear the transient state after draft_forward.

Constraint: topk > 1 reorders hidden rows in select_top_k_tokens, so carried topk indices would no longer match the hidden states.

Rejected: Reuse target-side topk for draft | broader semantic change not covered by the upstream fix or current tests.

Rejected: Skip loading draft indexer weights | separate memory optimization with correctness risk for models that do not enable MTP index sharing.

Confidence: high

Scope-risk: narrow

Directive: Do not enable index_share_for_mtp_iteration without preserving the topk==1 guard and per-draft-forward cleanup.

Tested: python -m pytest test/registered/spec/eagle/test_eagle_v2_draft_extend_contract.py -q

Tested: python -m py_compile python/sglang/srt/speculative/eagle_worker_v2.py python/sglang/srt/models/deepseek_nextn.py python/sglang/srt/model_executor/forward_batch_info.py test/registered/spec/eagle/test_eagle_v2_draft_extend_contract.py

Not-tested: full GLM5 EAGLE throughput/accuracy run
2026-06-29 03:16:14 +08:00
..
2026-06-10 05:54:43 +08:00