Files
sglang/python
leavelet b742371fe3 Skip get_load() snapshot when no DP load-balancer consumes it
stream_output_generation computed a full load snapshot on every
output flush: get_load() walks the waiting/bootstrap/prealloc queues
summing seqlens and builds a DpRequestInfoqOutput per queued request.
The only consumer is the DP load-balancer path in tokenizer_manager,
which forwards it iff dp_size > 1. Gate the computation on the same
condition; at dp_size == 1 the snapshot was computed and shipped for
nothing on every flush.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 05:12:40 +00:00
..
2026-06-10 05:54:43 +08:00