Files
sglang/docs/advanced_features
laoyao0822 e6ab699f8f Record CE-first direction for CP shared-KV L1 prefetch
The L1 prefetch design needs a clear boundary between SM IPC copies, copy-engine transport, and symmetric-memory address management. This updates the plan with the verified CE baseline, SYMM window prototype result, and the decision to prefer IPC CE for near-term prefetch integration while keeping SYMM as the future transport contract.

Constraint: Current PyTorch symmetric-memory prototype works but does not outperform IPC CE in remote benchmark sanity.

Rejected: Directly wire SYMM CE into SGLang hot path | benchmark showed SYMM CE 0.521ms gpu_p50 versus IPC CE 0.447ms for the same 8GPU fp8 sanity case.

Confidence: medium

Scope-risk: narrow

Directive: Keep prefix/history CE prefetch separate from current suffix ready-counter semantics until CE signaling is explicitly designed.

Tested: Documentation-only commit; tai-kernel CE/SYMM tests and benchmark recorded in paired tai-kernel commit f2a78dc

Not-tested: Full SGLang ETE with CE prefetch enabled
(cherry picked from commit 036662a9e2f907b527b6944c0ae4ac32e5d7618d)
2026-06-13 00:58:47 +08:00
..