chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

Gate scheduler metrics on attn_cp_rank==0 too, not just attn_tp_rank==0

Browse Source 
Under CP, attn_cp_size shards the TP group so compute_dp_attention_world_info
collapses attn_tp_size to 1 -> attn_tp_rank == 0 on EVERY CP rank. So
is_stats_logging_rank and current_scheduler_metrics_enabled were True on all
attn_cp_size ranks, making each rank log a 'Prefill batch' line and emit
scheduler-side metrics (SchedulerStats gauges, realtime token throughput) ->
summed token-throughput inflated by cp_size and duplicated per-rank logs.

Add attn_cp_rank == 0 to both gates, matching the scheduler's other single-rank
guards (scheduler.py:633/1591/1615/1631). No-op when CP is off (attn_cp_rank is
always 0). init_metrics runs after attn_cp_rank/attn_tp_rank are set
(scheduler.py:375/415 -> 433). Does NOT affect sglang:num_requests_total (a
TokenizerManager counter, one process per engine).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
孙远航
2026-06-25 06:27:52 +00:00
parent 01d6ceff8f
commit ce0e486327
1 changed files with 9 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
python/sglang/srt/observability/scheduler_metrics_mixin.py
View File

@@ -116,9 +116,16 @@ class SchedulerMetricsMixin:
# Metrics
self.enable_metrics = self.server_args.enable_metrics
self.is_stats_logging_rank = self.attn_tp_rank == 0
# Under CP, attn_cp_size shards the TP group so attn_tp_size collapses to 1 and
# attn_tp_rank == 0 on EVERY CP rank. Gate on (attn_tp_rank == 0 AND attn_cp_rank == 0)
# -- matching the scheduler's other single-rank guards -- so exactly one rank logs /
# emits scheduler-side metrics; otherwise all attn_cp_size ranks would each report,
# inflating summed token-throughput / SchedulerStats by cp_size. No-op when CP is off
# (attn_cp_rank is always 0 there).
self.is_stats_logging_rank = self.attn_tp_rank == 0 and self.attn_cp_rank == 0
self.current_scheduler_metrics_enabled = self.enable_metrics and (
self.attn_tp_rank == 0 or self.server_args.enable_metrics_for_all_schedulers
(self.attn_tp_rank == 0 and self.attn_cp_rank == 0)
or self.server_args.enable_metrics_for_all_schedulers
)
if self.enable_metrics:
if self.server_args.disaggregation_mode == DisaggregationMode.PREFILL.value: