Files
sglang/benchmark
laoyao0822 a149289554 Ground CP shared KV collective choices in production-shaped evidence
Add a production-style CP shared KV collective benchmark and keep the page-aligned cache contract ledger current with the collective, unpack, multi-extend prefix, P2P, and CUDA IPC findings. The benchmark makes the final consumer layout explicit so dense all-reduce and owner-packed all-gather are compared after producing the same logical-dense result.\n\nConstraint: Prefixes may be assembled from multiple radix/cache extents, so single-run zigzag ordering is only a gated fast path.\nRejected: Treat rank-major all-gather output as the final product | consumers require logical-dense ordering.\nRejected: Switch index materialization based on noisy microbenchmarks | current evidence is not strong enough for production.\nConfidence: medium\nScope-risk: narrow\nDirective: Do not replace generic logical-dense materialization with zigzag ordered gather unless a run descriptor proves the prefix shape.\nTested: python -m py_compile benchmark/hicache/bench_cp_shared_kv_production_collective.py\nTested: Remote production-style benchmark logs under /mnt/beegfs/cjy/log/cp_shared_kv_collective_bench_*_20260531_*.log\nNot-tested: Production SGLang ETE with a CUDA IPC/P2P materialization path; this commit only adds benchmark/documentation surfaces.
2026-05-31 17:04:00 +08:00
..
2025-09-24 02:35:25 +08:00
2024-05-05 16:14:17 +08:00
2026-02-16 00:18:31 +08:00
2026-02-13 18:08:57 +08:00