chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

Reduce CP per-layer transfer success-log noise

Browse Source 
Per-layer prefill-to-decode transfer now finishes per request/rank/chunk, so success-path INFO logs can dominate production logs and hide actual failures. Keep successful finish breakdown and completion messages at DEBUG while preserving nonzero finish status as WARNING.

Constraint: Per-layer transfer is a hot path under CP shared-KV and may produce many batch completions per request.

Rejected: Disable CP per-layer transfer logging entirely | failures still need visible warning-level evidence.

Confidence: high

Scope-risk: narrow

Directive: Do not promote successful per-request transfer completion logs back to INFO without rate limiting.

Tested: PYTHONPATH=python python -m pytest -q test/registered/unit/disaggregation/test_cp_per_layer_transfer.py::TestPerLayerTransferContext::test_successful_finish_does_not_emit_hot_path_info_log

Tested: python -m py_compile python/sglang/srt/disaggregation/cp_per_layer_transfer.py python/sglang/srt/disaggregation/mooncake/conn.py

Not-tested: Full local disaggregation suite blocked by missing local orjson dependency.

Co-authored-by: OmX <omx@oh-my-codex.dev>
This commit is contained in:
laoyao0822
2026-06-11 05:49:05 +08:00
parent 7284a469a2
commit 8cfdb0466e
3 changed files with 30 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
python/sglang/srt/disaggregation/cp_per_layer_transfer.py
View File

@@ -186,8 +186,8 @@ class PerLayerTransferContext:
batch_ids = list(self._batch_ids)
wait_status = self.engine.wait_batch_transfers(batch_ids)
t3 = time.perf_counter() # RDMA complete
if logger.isEnabledFor(logging.INFO):
logger.info(
if logger.isEnabledFor(logging.DEBUG):
logger.debug(
"[CP_PER_LAYER_TRANSFER] finish breakdown: target=%d submitted_at_entry=%d "
"missing=%d wait_workers=%.1fms fallback=%.1fms wait_rdma=%.1fms total=%.1fms",
target,

17
python/sglang/srt/disaggregation/mooncake/conn.py
View File

@@ -1280,11 +1280,18 @@ class MooncakeKVManager(CommonKVManager):
# with the forward; wait those transfers here instead of the
# monolithic send (no double-send). aux/state below unchanged.
ret = per_layer_mgr.finish(kv_chunk.room)
logger.info(
"[CP_PER_LAYER_TRANSFER] finished room=%s ret=%s",
kv_chunk.room,
ret,
)
if ret != 0:
logger.warning(
"[CP_PER_LAYER_TRANSFER] finished room=%s ret=%s",
kv_chunk.room,
ret,
)
else:
logger.debug(
"[CP_PER_LAYER_TRANSFER] finished room=%s ret=%s",
kv_chunk.room,
ret,
)
elif self.is_mla_backend or (
self.attn_tp_size
== target_rank_registration_info.dst_attn_tp_size

16
test/registered/unit/disaggregation/test_cp_per_layer_transfer.py
View File

@@ -10,6 +10,7 @@ Key correctness properties:
Engine and CUDA events are mocked — no GPU/RDMA.
"""
import unittest
from unittest import mock
from sglang.srt.disaggregation.cp_per_layer_transfer import PerLayerTransferContext
@@ -128,6 +129,21 @@ class TestPerLayerTransferContext(unittest.TestCase):
self.assertEqual(eng.waits[-1], [101, 102])
self.assertEqual(eng.submits[1], ("sess", [1000, 2000], [6000, 7000], [64, 64]))
def test_successful_finish_does_not_emit_hot_path_info_log(self):
# Successful per-request transfer completion is hot-path and can fire once per
# request/rank/chunk. It must not emit INFO logs by default.
eng = _FakeEngine()
ctx = _ctx(eng, num_layers=1)
ctx.submit_layer(0, None)
with mock.patch(
"sglang.srt.disaggregation.cp_per_layer_transfer.logger.isEnabledFor",
return_value=True,
), mock.patch(
"sglang.srt.disaggregation.cp_per_layer_transfer.logger.info"
) as info:
self.assertEqual(ctx.finish(timeout=0.5), 0)
info.assert_not_called()
def test_failed_submit_still_reported_after_fallback(self):
# layer 2 submit fails during the overlap; finish reports -1 even though it
# also runs the fallback for any unfired layers.