chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Reduce CP per-layer transfer success-log noise

Browse Source 
Per-layer prefill-to-decode transfer now finishes per request/rank/chunk, so success-path INFO logs can dominate production logs and hide actual failures. Keep successful finish breakdown and completion messages at DEBUG while preserving nonzero finish status as WARNING.

Constraint: Per-layer transfer is a hot path under CP shared-KV and may produce many batch completions per request.

Rejected: Disable CP per-layer transfer logging entirely | failures still need visible warning-level evidence.

Confidence: high

Scope-risk: narrow

Directive: Do not promote successful per-request transfer completion logs back to INFO without rate limiting.

Tested: PYTHONPATH=python python -m pytest -q test/registered/unit/disaggregation/test_cp_per_layer_transfer.py::TestPerLayerTransferContext::test_successful_finish_does_not_emit_hot_path_info_log

Tested: python -m py_compile python/sglang/srt/disaggregation/cp_per_layer_transfer.py python/sglang/srt/disaggregation/mooncake/conn.py

Not-tested: Full local disaggregation suite blocked by missing local orjson dependency.

Co-authored-by: OmX <omx@oh-my-codex.dev>
This commit is contained in:
laoyao0822
2026-06-11 05:49:05 +08:00
parent 7284a469a2
commit 8cfdb0466e
3 changed files with 30 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
python/sglang/srt/disaggregation/cp_per_layer_transfer.py
View File

@@ -186,8 +186,8 @@ class PerLayerTransferContext:
batch_ids = list(self._batch_ids)
wait_status = self.engine.wait_batch_transfers(batch_ids)
t3 = time.perf_counter() # RDMA complete
if logger.isEnabledFor(logging.INFO):
logger.info(
if logger.isEnabledFor(logging.DEBUG):
logger.debug(
"[CP_PER_LAYER_TRANSFER] finish breakdown: target=%d submitted_at_entry=%d "
"missing=%d wait_workers=%.1fms fallback=%.1fms wait_rdma=%.1fms total=%.1fms",
target,

17
python/sglang/srt/disaggregation/mooncake/conn.py
View File

@@ -1280,11 +1280,18 @@ class MooncakeKVManager(CommonKVManager):
# with the forward; wait those transfers here instead of the
# monolithic send (no double-send). aux/state below unchanged.
ret = per_layer_mgr.finish(kv_chunk.room)
logger.info(
"[CP_PER_LAYER_TRANSFER] finished room=%s ret=%s",
kv_chunk.room,
ret,
)
if ret != 0:
logger.warning(
"[CP_PER_LAYER_TRANSFER] finished room=%s ret=%s",
kv_chunk.room,
ret,
)
else:
logger.debug(
"[CP_PER_LAYER_TRANSFER] finished room=%s ret=%s",
kv_chunk.room,
ret,
)
elif self.is_mla_backend or (
self.attn_tp_size
== target_rank_registration_info.dst_attn_tp_size