Reuse draft MTP indexer topk on spec v1 EAGLE

Spec v1 EAGLE now follows the same model-config-gated MTP index reuse contract as spec v2. This lets models that declare index_share_for_mtp_iteration reuse the first draft step's NSA/DSA topk indices across internal MTP iterations while keeping the unsafe topk>1 case disabled.

Constraint: select_top_k_tokens can reorder hidden rows when topk > 1, so carried topk indices are only valid under topk == 1.

Rejected: Enable reuse unconditionally | models without the config flag may not have compatible MTP index semantics.

Rejected: Broaden to target-to-draft index reuse | separate semantic change with different correctness risks.

Confidence: high

Scope-risk: narrow

Directive: Keep spec v1 and spec v2 MTP index reuse semantics aligned, including the topk==1 guard and per-draft-forward cleanup.

Tested: python -m pytest test/registered/spec/eagle/test_eagle_v2_draft_extend_contract.py -q

Tested: python -m py_compile python/sglang/srt/speculative/eagle_worker.py test/registered/spec/eagle/test_eagle_v2_draft_extend_contract.py

Not-tested: full spec v1 GLM5 online throughput run
This commit is contained in:
laoyao0822
2026-06-28 00:57:31 +08:00
parent df3cc9abf8
commit ac47eb61c9
2 changed files with 86 additions and 0 deletions

View File

@@ -156,6 +156,19 @@ class EAGLEWorker(TpModelWorker):
memory_pool_config=target_worker.model_runner.memory_pool_config,
)
# Reuse the first draft step's NSA/DSA indexer topk across the rest of
# the MTP iteration when the model config says it is safe. The reuse is
# only valid for topk == 1: select_top_k_tokens reorders rows for topk
# > 1, which would desynchronize carried indices from hidden states.
self.index_share_for_mtp_iteration = (
getattr(
self.draft_model_runner.model_config.hf_config,
"index_share_for_mtp_iteration",
False,
)
and self.topk == 1
)
embed, head = self.target_worker.model_runner.model.get_embed_and_head()
if self.speculative_algorithm.is_eagle3():
@@ -701,6 +714,10 @@ class EAGLEWorker(TpModelWorker):
token_list: List[torch.Tensor] = []
parents_list: List[torch.Tensor] = []
if self.index_share_for_mtp_iteration:
forward_batch.reuse_mtp_topk_indices = True
forward_batch.topk_indices = None
# Forward multiple steps
scores = None
for i in range(self.speculative_num_steps):
@@ -751,6 +768,10 @@ class EAGLEWorker(TpModelWorker):
score_list, token_list, parents_list, self.speculative_num_draft_tokens
)
if self.index_share_for_mtp_iteration:
forward_batch.topk_indices = None
forward_batch.reuse_mtp_topk_indices = False
return parent_list, top_scores_index, draft_tokens
def clear_cache_pool(self):