chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

Expose gated evidence for CP shared-KV bs>1 debugging

Browse Source 
The bs>1 prefill path has multiple coupled stages: scheduler admission, page-aligned batch planning, tensor splitting, direct cache writes, index top-k, MLA reuse, and disaggregated KV handoff. Add a default-off, rate-limited debug channel so production ETE runs can identify where batching or metadata semantics diverge without permanently increasing hot-path log volume.

Constraint: Logs must be default-off and rate-limited because these paths execute per-rank and often per-layer.

Rejected: Always-on INFO logs | would flood logs and add CPU overhead during normal prefill.

Rejected: Only scheduler-side logging | insufficient to distinguish planner, index, MLA, and transfer handoff failures.

Confidence: medium

Scope-risk: moderate

Directive: Keep bs>1 debug evidence env-gated; do not add unconditional per-layer or per-token logs in these paths.

Tested: Local py_compile for touched files

Tested: git diff --check

Tested: Remote py_compile and targeted NSA CP utility tests: 5 passed

Not-tested: Full ETE correctness with debug disabled
This commit is contained in:
laoyao0822
2026-06-05 23:48:08 +08:00
parent 6eea77e5e9
commit 1b99de7459
7 changed files with 326 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/sglang/srt/environ.py
+2
View File
@@ -203,6 +203,8 @@ class Envs:
SGLANG_FORCE_SHUTDOWN = EnvBool(False)
SGLANG_DEBUG_MEMORY_POOL = EnvBool(False)
SGLANG_DEBUG_CP_SHARED_KV = EnvBool(False)
SGLANG_CP_SHARED_KV_BS_GT1_DEBUG = EnvBool(False)
SGLANG_CP_SHARED_KV_BS_GT1_DEBUG_LIMIT = EnvInt(128)
SGLANG_DEBUG_SORT_NVTX = EnvBool(False)
SGLANG_DEBUG_MOE_SORT_NVTX = EnvBool(False)
SGLANG_CP_SHARED_KV_CURRENT_REUSE = EnvBool(False)