Record CUDA IPC as CP compose main-stream choice

The symm-vs-IPC matrix showed CUDA IPC winning most production-shaped cases, especially bs=5/10, so the formal compose path should stay on CUDA IPC while symm remains an experimental/prefetch direction. Record the benchmark artifacts and decision to avoid re-litigating the branch choice during future rebases.\n\nConstraint: Production CP shared-KV compose must favor the measured low-risk path, not a speculative symm route.\nRejected: Make symm the main compose path | benchmark won only 3/24 cases and adds a separate synchronization contract.\nConfidence: high\nScope-risk: narrow\nDirective: Do not reintroduce symm writer-rank/barrier parameters into main nsa_backend/nsa_indexer compose without a newer benchmark that beats CUDA IPC end-to-end.\nTested: Documentation-only change; benchmark artifacts recorded from g0034 run.\nNot-tested: No service restart required.
This commit is contained in:
laoyao0822
2026-06-13 01:17:50 +08:00
parent 254d667853
commit f290e1e35c

View File

@@ -440,3 +440,35 @@ bs=2 cached=102k: all_reduce ~0.81ms, descriptor setup ~1.05ms, runtime IPC ~1.6
远端 cjy-glm5-newPYTHONPATH=python python -m pytest -q test/registered/unit/mem_cache/test_cp_shared_kv_runtime.py
146 passed, 21 warnings, 2 subtests passed
```
### 2026-06-13 symm vs CUDA IPC 主路径选择
`g0034 / cjy-glm5-new / 8xH200` 上补齐了 symm kernel 与 CUDA IPC kernel 的 24-case 对比矩阵:
- 脚本:`/mnt/beegfs/cjy/run_cp_compose_matrix.py`
- 原始日志:`/mnt/beegfs/cjy/bench_results/cp_compose_symm_vs_ipc_20260612_163953.log`
- JSONL`/mnt/beegfs/cjy/bench_results/cp_compose_symm_vs_ipc_20260612_163953.jsonl`
- 汇总:`/mnt/beegfs/cjy/bench_results/cp_compose_symm_vs_ipc_20260612_163953.md`
矩阵覆盖:
- bs=1/2/5/10。
- cached prefix 约 65k/102k/160k/190k。
- extend 约 256/1024/4096/32768/65536。
- page_size=64warmup=2repeat=5no-check。
- 对比 `d_ipc_all_pages``f_staged_symm_mega`
关键结论:
1. CUDA IPC 赢 21/24symm 只赢 3/24。
2. 平均相对 fused all_reduce 收益CUDA IPC 约 1.95xsymm mega 约 1.92x。
3. symm mega 平均比 CUDA IPC 慢约 1.86%,最差约 5.62%。
4. bs=5/10 下 CUDA IPC 全胜。
5. 非 mega staged symm 平均慢约 17%,不适合作为 production materialize 主路径。
6. symm barrier GPU 时间通常约 0.018-0.020ms,偶发约 0.048ms;同步成本不算灾难性,但也没有带来足够收益抵消主路径风险。
因此当前 production main-stream compose 决策:
- 主路径继续使用 CUDA IPC kernel。
- symm kernel 保留为 benchmark / 实验路径,以及未来 0SM/CE prefetch 方向的参考。
- 不把 symm writer-rank / barrier 参数接入正式 `nsa_backend.py` / `nsa_indexer.py` compose 调用,避免引入额外同步合同和 rebase 接口分裂。