Skip get_load() snapshot when no DP load-balancer consumes it

stream_output_generation computed a full load snapshot on every
output flush: get_load() walks the waiting/bootstrap/prealloc queues
summing seqlens and builds a DpRequestInfoqOutput per queued request.
The only consumer is the DP load-balancer path in tokenizer_manager,
which forwards it iff dp_size > 1. Gate the computation on the same
condition; at dp_size == 1 the snapshot was computed and shipped for
nothing on every flush.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-06-10 05:12:40 +00:00
parent e6864fbea7
commit b742371fe3

View File

@@ -1066,7 +1066,11 @@ class SchedulerOutputProcessorMixin:
spec_acceptance_histogram = []
retraction_counts = []
output_hidden_states = None
load = self.get_load()
# The load snapshot is only consumed by the DP load-balancer
# (tokenizer_manager forwards it iff dp_size > 1). get_load() walks
# every queue and builds per-request info objects, so skip it entirely
# when nothing will read it.
load = self.get_load() if self.server_args.dp_size > 1 else None
routed_experts = None
customized_info = {}