Files
sglang/python
laoyao0822 f20ef7ace4 Avoid redundant CP KV rebuild on shared-KV MLA path
Shared-KV prefill already persists each rank's MLA shard and reconstructs the dense attention KV from the shared pool before attention. Keeping the legacy CP rebuild all-gather after direct write duplicated communication on the hot MQA-to-attention path. The rebuild remains enabled only for current-only reuse, where the backend intentionally consumes full current KV tensors instead of materializing history from the pool. Index and MLA prefetch now share one FIFO CUDA stream so their CP collectives preserve local launch order.

Constraint: CP shared-KV materialize is the authoritative KV source for prefix/cache-hit MLA attention.

Rejected: Gate prefetch by prefix owner-lane coverage | owner skew is not the root cause and would add an extra collective plus CPU sync.

Confidence: medium

Scope-risk: moderate

Directive: Do not reintroduce rebuild_cp_kv_cache for shared-KV direct-write unless the backend consumes k_nope/k_pe directly.

Tested: git diff --check; py_compile for cp_shared_kv_prefetch.py, nsa_backend.py, forward_mla.py; remote container py_compile after scp sync.

Not-tested: Full multi-node GLM5 performance run after this exact commit.
2026-05-12 20:20:33 +08:00
..