From f290e1e35c1733f3cc9cf7a1f61fe3705fb47e16 Mon Sep 17 00:00:00 2001
From: laoyao0822 <laoyao0822@qq.com>
Date: Sat, 13 Jun 2026 01:17:50 +0800
Subject: [PATCH] Record CUDA IPC as CP compose main-stream choice

The symm-vs-IPC matrix showed CUDA IPC winning most production-shaped cases, especially bs=5/10, so the formal compose path should stay on CUDA IPC while symm remains an experimental/prefetch direction. Record the benchmark artifacts and decision to avoid re-litigating the branch choice during future rebases.\n\nConstraint: Production CP shared-KV compose must favor the measured low-risk path, not a speculative symm route.\nRejected: Make symm the main compose path | benchmark won only 3/24 cases and adds a separate synchronization contract.\nConfidence: high\nScope-risk: narrow\nDirective: Do not reintroduce symm writer-rank/barrier parameters into main nsa_backend/nsa_indexer compose without a newer benchmark that beats CUDA IPC end-to-end.\nTested: Documentation-only change; benchmark artifacts recorded from g0034 run.\nNot-tested: No service restart required.
---
 ...d_kv_ipc_collective_replacement_plan_zh.md | 32 +++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/docs/advanced_features/nsa_prefill_cp_shared_kv_ipc_collective_replacement_plan_zh.md b/docs/advanced_features/nsa_prefill_cp_shared_kv_ipc_collective_replacement_plan_zh.md
index 29a45329d..ea5f921d2 100644
--- a/docs/advanced_features/nsa_prefill_cp_shared_kv_ipc_collective_replacement_plan_zh.md
+++ b/docs/advanced_features/nsa_prefill_cp_shared_kv_ipc_collective_replacement_plan_zh.md
@@ -440,3 +440,35 @@ bs=2 cached=102k: all_reduce ~0.81ms, descriptor setup ~1.05ms, runtime IPC ~1.6
 远端 cjy-glm5-new：PYTHONPATH=python python -m pytest -q test/registered/unit/mem_cache/test_cp_shared_kv_runtime.py
 146 passed, 21 warnings, 2 subtests passed
 ```
+
+### 2026-06-13 symm vs CUDA IPC 主路径选择
+
+在 `g0034 / cjy-glm5-new / 8xH200` 上补齐了 symm kernel 与 CUDA IPC kernel 的 24-case 对比矩阵：
+
+- 脚本：`/mnt/beegfs/cjy/run_cp_compose_matrix.py`
+- 原始日志：`/mnt/beegfs/cjy/bench_results/cp_compose_symm_vs_ipc_20260612_163953.log`
+- JSONL：`/mnt/beegfs/cjy/bench_results/cp_compose_symm_vs_ipc_20260612_163953.jsonl`
+- 汇总：`/mnt/beegfs/cjy/bench_results/cp_compose_symm_vs_ipc_20260612_163953.md`
+
+矩阵覆盖：
+
+- bs=1/2/5/10。
+- cached prefix 约 65k/102k/160k/190k。
+- extend 约 256/1024/4096/32768/65536。
+- page_size=64，warmup=2，repeat=5，no-check。
+- 对比 `d_ipc_all_pages` 与 `f_staged_symm_mega`。
+
+关键结论：
+
+1. CUDA IPC 赢 21/24，symm 只赢 3/24。
+2. 平均相对 fused all_reduce 收益：CUDA IPC 约 1.95x，symm mega 约 1.92x。
+3. symm mega 平均比 CUDA IPC 慢约 1.86%，最差约 5.62%。
+4. bs=5/10 下 CUDA IPC 全胜。
+5. 非 mega staged symm 平均慢约 17%，不适合作为 production materialize 主路径。
+6. symm barrier GPU 时间通常约 0.018-0.020ms，偶发约 0.048ms；同步成本不算灾难性，但也没有带来足够收益抵消主路径风险。
+
+因此当前 production main-stream compose 决策：
+
+- 主路径继续使用 CUDA IPC kernel。
+- symm kernel 保留为 benchmark / 实验路径，以及未来 0SM/CE prefetch 方向的参考。
+- 不把 symm writer-rank / barrier 参数接入正式 `nsa_backend.py` / `nsa_indexer.py` compose 调用，避免引入额外同步合同和 rebase 接口分裂。