Record compact IPC staging capacity contract
The index current IPC startup failure came from a tai-kernel validation mismatch: compact staging allocations are rounded capacity buffers and do not need to be page-size divisible. Record the failure signature and the intended contract so future changes do not reintroduce dense-sized or page-multiple assumptions. Constraint: TAI compact staging fix lives in the kernel repository; this change documents the runtime contract in SGLang Rejected: Leave the issue only in commit history | future debugging would repeat the log search and root-cause analysis Confidence: high Scope-risk: narrow Directive: For compact current staging, validate capacity bytes, not page-array shape Tested: Remote cjy-glm5-new SGLang runtime helper suite -> 144 passed, 2 subtests passed Not-tested: Documentation rendering (cherry picked from commit ce281c19be5d54cd2cb345543b0384d7560c95ef)
This commit is contained in:
@@ -320,9 +320,35 @@ peer read staging[compact_i] -> dense_current[dense_page_id_i]
|
||||
3. 不允许回到静默 all_reduce fallback。CUDA + TAI materialize 开启时,compact IPC 不可用仍应 fail-fast。
|
||||
4. 后续 fused current fill+publish 仍有价值:compact staging 解决容量问题,但 current 仍有一次额外 HBM copy。
|
||||
|
||||
### 2026-06-12 index current 启动失败补充
|
||||
|
||||
启动阶段又触发:
|
||||
|
||||
```text
|
||||
[CP_SHARED_KV_FAIL_FAST][tai_ipc_materialize]
|
||||
reason=index_current_ipc_unavailable
|
||||
dense_shape=(2, 8448)
|
||||
```
|
||||
|
||||
前置 warning:
|
||||
|
||||
```text
|
||||
reason=current_paged_kernel_failed
|
||||
error=staging byte size must be divisible by page_nbytes
|
||||
```
|
||||
|
||||
这是 compact publish op 的 C++ check 错误,不是 runtime descriptor 错误。index page bytes 为 8448,current staging cache 会按 power-of-two 扩容到 16384;16384 不能被 8448 整除,但这完全合法。compact staging 只要求:
|
||||
|
||||
```text
|
||||
staging_nbytes >= num_slots * page_nbytes
|
||||
```
|
||||
|
||||
不要求整个 staging allocation 被 `page_nbytes` 整除。TAI 单测已覆盖 `page_nbytes=8448, staging_nbytes=16384`。
|
||||
|
||||
### 验证
|
||||
|
||||
- 新增 TAI CUDA 单测:dense 10 pages、staging 3 pages,只发布 `[1, 4, 9]` 到 compact staging,验证不需要 dense-sized staging。
|
||||
- 新增 TAI CUDA 单测:`page_nbytes=8448`、`staging_nbytes=16384`,验证 power-of-two staging capacity 可用于 index current page。
|
||||
- SGLang runtime 单测覆盖 token/index current helper:publish 使用 dense destination pages,peer materialize 使用 compact source pages。
|
||||
- `benchmark_cp_shared_kv_ipc_gather.py` 已同步改成 compact current staging 合同;quick smoke:
|
||||
`bs=2 cached=4096 extend=1024 fp8/uint8` 下 dense all_reduce p50 0.459ms,IPC compose p50 0.310ms。
|
||||
|
||||
Reference in New Issue
Block a user