Files
sglang/python
laoyao0822 4d5c7f32d6 Keep CP shared-KV prefetch warnings actionable
Expected no-prefetch paths were polluting production logs: no cache prefix, tiny/first-layer windows, and FP8 RAGGED top-k were being reported as fallback warnings. The prefetch contract now treats zero-prefix and first-layer misses as normal skips, while preserving warnings for non-zero misaligned prefixes and real consume misses after the first layer. The same change keeps RAGGED cache-hit prefetch eligible and records the CE/IPM prefetch contract in the plan doc.

Constraint: FP8 sparse prefill uses RAGGED top-k, but CP shared-KV prefix materialization is still page-slot based

Constraint: Layer 0 has no previous attention-window hook that can have prefetched the layer

Rejected: Warn whenever a prefetcher is absent | no-cache and too-short requests are expected synchronous paths and make logs unusable

Confidence: high

Scope-risk: moderate

Directive: Keep CP_SHARED_KV_FALLBACK warnings for unexpected contract failures only; use debug logs for expected skip paths

Tested: Local py_compile for cp_shared_kv_prefetch.py, nsa_indexer.py, nsa_backend.py

Tested: Remote cjy-glm5-new targeted regression: 3 passed, 21 warnings

Tested: Remote cjy-glm5-new full test_cp_shared_kv_runtime.py: 156 passed, 21 warnings, 2 subtests passed

Not-tested: New ETE run after prefill restart to confirm log volume reduction in production traffic
(cherry picked from commit e08e321e5929fdbb30102ec0b19c6ff0ecac7e7e)
2026-06-13 00:59:33 +08:00
..
2026-06-10 05:54:43 +08:00