Files
sglang/python
leavelet 2b8593ff24 B300 NSA: default NSA prefill to flashmla under context parallelism (fix k_rope CP crash)
_set_default_nsa_backends defaulted CP prefill to trtllm on Blackwell+fp8-KV, but the trtllm NSA
prefill path mis-shards k_rope under CP: the rope input carries all CP ranks' tokens (nnz*cp) while
the per-rank rope kernel asserts k_rope_in.size(0)==nnz -> 'Check failed: 256 vs 64' at cp=4. The
function's own comment already notes flashmla_sparse pads heads 64->128 and works under CP; this just
auto-selects it under CP instead of requiring an explicit --nsa-prefill-backend override. Matches the
gov journal (2026-06-08 entry: trtllm-for-CP-prefill was the wrong backend). Decode/non-CP unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 17:33:49 +00:00
..
2026-06-10 05:54:43 +08:00