fix: copy spec_info in ScheduleBatch.copy() to preserve Eagle hidden states in overlap prefill
In disaggregated prefill with --enable-single-batch-overlap, batch.copy() was used to defer process_batch_result, but the copy omitted spec_info. This caused req.hidden_states_tensor to be None when using Eagle/spec-v2 speculative decoding on the decode server, silently degrading draft quality for the first decode round.
This commit is contained in:
@@ -2343,6 +2343,7 @@ class ScheduleBatch(ScheduleBatchDisaggregationDecodeMixin):
|
||||
return_logprob=self.return_logprob,
|
||||
decoding_reqs=self.decoding_reqs,
|
||||
spec_algorithm=self.spec_algorithm,
|
||||
spec_info=self.spec_info,
|
||||
global_num_tokens=self.global_num_tokens,
|
||||
global_num_tokens_for_logprob=self.global_num_tokens_for_logprob,
|
||||
can_run_dp_cuda_graph=self.can_run_dp_cuda_graph,
|
||||
|
||||
Reference in New Issue
Block a user