Files
sglang/python
laoyao0822 142e7a5a64 Keep CP shared-KV fast paths off dense current collectives
CP shared-KV cache-hit batches should compose long prefix pages and short current pages through page-slot IPC instead of falling back to dense all_reduce. Wire the runtime and prefetch consume paths to the TAI current-staging helpers, fail fast when the configured CUDA fast path cannot run, and document the bs>1 cache-hit benchmark evidence.

Constraint: bs>1 prefill must preserve the page-slot contract across fp8/bf16 and zero-lane current tails.

Rejected: Silent all_reduce fallback | hides correctness and performance regressions in production.

Confidence: medium

Scope-risk: moderate

Directive: Any future fallback in CP shared-KV CUDA fast paths must be explicit warning/fail-fast and covered by runtime tests.

Tested: Local py_compile cp_shared_kv_runtime.py and cp_shared_kv_prefetch.py; remote PYTHONPATH=python pytest -q test/registered/unit/mem_cache/test_cp_shared_kv_runtime.py (144 passed, 21 warnings, 2 subtests passed); remote TAI IPC benchmark fp8 bs>1 cache-hit matrix recorded in docs.

Not-tested: Full ETE mixed replay after replacing all current collectives with IPC.
(cherry picked from commit 8aa3b4ce59e0ebef5da5b0d07499a5f1d9785997)
2026-06-13 00:58:47 +08:00
..
2026-06-10 05:54:43 +08:00