Files
sglang/python
laoyao0822 b324407def Trace CP HiCache cache-hit state for GSM8K regressions
The cache-hit accuracy drop is not isolated to one obvious kernel boundary, so the scheduler/radix/HiCache path needs request-correlated evidence.  Add default-off, rate-limited CP shared-KV debug logs for prefix matching, valid-tail flooring, pending backup attach/commit, load-back planning, and unfinished-request cache insertion.  The investigation notes capture current GSM8K pass1/pass2 evidence and rejected interpretations to avoid repeated log archaeology.

Constraint: Logs must be default-off and bounded because these paths are scheduler/control hot paths.

Rejected: Always-on INFO tracing | would distort the exact latency and cache-hit behavior under investigation.

Confidence: medium

Scope-risk: moderate

Directive: Remove or keep env-gated only after the GSM8K cache-hit root cause is closed; do not leave unbounded hot-path logs.

Tested: python -m py_compile on changed runtime files.

Not-tested: Local pytest blocked before collection by missing orjson dependency; fresh restarted GSM8K verification still pending.
(cherry picked from commit bfc6d1473dc2a5d72bc3a8d6fca1e2429537be0e)
2026-06-08 00:17:28 +08:00
..