Files
sglang/python
laoyao0822 b7364d23f9 Keep draft cache-hit KV on current-suffix compose
Cache-hit EAGLE/NextN draft was falling back to full materialization while the target path used page-aligned prefix materialize plus fresh current-suffix splice. That creates a target/draft KV source asymmetry exactly on the high-cache-hit path where decode accept length collapsed. The draft model still does not get next-layer async prefetch; only the same-layer current suffix compose contract is made role-agnostic.

Constraint: EAGLE/NextN has no real next decoder layer, so draft async prefetch remains disabled.

Rejected: Restore draft full-materialize fallback | recreates the observed target/draft cache-hit asymmetry and hides stale-current-suffix bugs.

Confidence: medium

Scope-risk: moderate

Directive: Do not reintroduce draft cache-hit current-reuse fallback without proving the draft persistent pool has fully fresh suffix rows before attention.

Tested: Local py_compile for cp_shared_kv_runtime.py and test_cp_shared_kv_runtime.py.

Tested: Remote g0034 container py_compile for changed runtime/test files.

Tested: Remote g0034 pytest test_cp_shared_kv_runtime.py: 73 passed, 5 warnings, 2 subtests passed.

Tested: Remote g0034 related suite test_nsa_cp_utils.py test_cp_shared_kv_layout.py test_cp_shared_kv_runtime.py: 128 passed, 5 warnings, 2 subtests passed.

Not-tested: ETE cache-hit accept-length validation after restarting prefill/decode with this synced code.
2026-05-30 03:06:17 +08:00
..