Files
sglang/python
laoyao0822 bc9a9f128b perf(cp): add narrow output collection for NSA prefill CP
Replace full hidden all-gather at prefill tail with per-request last-token
hidden collection, reducing communication from total_tokens x hidden_size
to cp_size x bs x hidden_size for both in-seq-split and round-robin modes.

- nsa/utils.py: add cp_collect_last_token_hidden() with mode-specific
  narrow collection helpers that only gather the last token hidden
- deepseek_v2.py: add _should_use_narrow_output_path() gate on
  DeepseekV2Model, fallback to full gather for EAGLE/return_logprob/
  capture_hidden batches
- logits_processor.py: add _is_compact_hidden_states() to bypass
  _get_pruned_states() when hidden is already compact
2026-04-22 04:55:06 +08:00
..