Preserve CP narrow output while planning real batches

Batch-size support needs request-first CP metadata; treating a batch as one long sequence breaks page ownership, top-k ranges, and phase1 compact output collection. This adds a batch CP plan that records per-request page-aligned splits, rank-local offsets, kv/actual-seq metadata, last-token owners, and flattened descriptors for downstream allocator/runtime workstreams.

The scalar full-rerange path now fail-fasts for batch metadata so bs>1 cannot silently discard the narrow-output optimization or restore hidden states with single-request assumptions.

Constraint: CP shared-KV cache state is page-owned and must preserve request boundaries under bs>1.

Rejected: Let bs>1 fall back to scalar full hidden rerange | it loses the phase1 communication reduction and uses wrong single-request metadata.

Rejected: Add a collective to confirm batch plans | all ranks can derive the same plan from CPU metadata and config.

Confidence: medium

Scope-risk: moderate

Directive: Do not remove batch fail-fast guards until W2/W3 consumers use CPSharedKVBatchPlan end-to-end.

Tested: python -m py_compile python/sglang/srt/layers/attention/nsa/utils.py test/registered/unit/layers/test_nsa_cp_utils.py

Tested: remote g0034 container PYTHONPATH=python python -m pytest -q test/registered/unit/layers/test_nsa_cp_utils.py -> 39 passed

Not-tested: full ETE bs>1 CP shared-KV runtime; W2/W3 allocator/direct-write consumers are not implemented yet
This commit is contained in:
laoyao0822
2026-06-03 01:21:43 +08:00
parent 0158e28689
commit e4cf8d18b4
4 changed files with 812 additions and 15 deletions

View File

@@ -272,8 +272,15 @@ NSAContextParallelMetadata
request_kv_len_next: List[int]
request_actual_seq_q_prev: List[int]
request_actual_seq_q_next: List[int]
request_actual_seq_q_prev_cu_tensor: Tensor[batch_size + 1]
request_actual_seq_q_next_cu_tensor: Tensor[batch_size + 1]
request_rank_local_offsets: List[int]
request_last_token_owner: List[int]
request_last_token_local_offset: List[int]
flat_split_list: List[int]
flat_zigzag_index: List[int]
flat_segment_request_ids: List[int]
flat_segment_offsets: List[int]
```
兼容规则:
@@ -347,9 +354,14 @@ build_batch_page_aligned_in_seq_split_plan(
- per-request split list
- per-request page start/end
- per-rank local valid token count
- per-request `kv_len_prev/next``actual_seq_q_prev/next`
- per-request last-token owner/local offset
- per-request rank-local offset用于 batch compact hidden collect
- flattened segment offsets。
4. 更新 `prepare_input_dp_with_cp_dsa()`,让 CP shared-KV bs>1 使用新 metadata。
5. 保留 bs=1 的旧字段兼容。
6. `cp_collect_last_token_hidden()` 必须继续走 phase1 compact/narrow output不能因为 bs>1 回到 full hidden gather。
7. bs>1 full rerange 在 batch-aware full output metadata/kernels 完成前 fail-fast。
测试重点:
@@ -371,6 +383,9 @@ cp_size=8
退出标准:
- metadata 能表达 `batch_size=2`,且不会把两个 request 当成一条长序列。
- 普通 prefill bs>1 的 output collection 能按 request order 返回 compact last hidden。
- 需要 full output 语义的 bs>1 请求不会误用 scalar full rerange。
- 对外 helper 能提供 batch plan、按 plan split tensor、以及 flat page-owner plan。
### Phase 2batch-aware owner-lane allocation

View File

@@ -146,6 +146,14 @@ CPSharedKVBatchPlan
request_actual_seq_q_prev
request_actual_seq_q_next
request_rank_local_tokens
request_rank_local_offsets
request_actual_seq_q_prev_cu_tensor
request_actual_seq_q_next_cu_tensor
# phase1 narrow output collection
request_last_token_owner
request_last_token_local_offset
output_collect_mode # narrow_last_token | full_rerange | unsupported_fail_fast
flat_split_list
flat_zigzag_index
@@ -161,6 +169,8 @@ CPSharedKVBatchPlan
4. zero-token segment 合法。
5. batch flattened plan 不能跨 request 合并 segment。
6. bs=1 继续兼容现有 scalar fields。
7. phase1 narrow-output 优化不能因为 bs>1 回退:普通 prefill 必须通过 `request_last_token_owner/local_offset` 只收集每个 request 的 last hidden。
8. bs>1 的 full rerange/logprob/hidden capture 在 batch-aware full rerange 完成前必须 fail-fast不能使用 scalar `cp_all_gather_rerange_output()`
### 对外接口