Files
sglang/test/manual
leavelet 9d65bdba95 Measure mqa-logits chunk pipelining: not worth it (S2b closed)
The bench gains a pipelined mode (topk_N event-gated on a side stream,
overlapping logits_{N+1}).  Byte-equality validation was dropped after
establishing the kernel chain is run-to-run nondeterministic even
serial-vs-serial (near-equal fp32 selection).

Result on g0033 H200: pipelining recovers only 0.3-8.9% where small
chunks cost +15-46% — fp8_mqa_logits and fast_topk_transform_fused are
both SM-saturating, so concurrent streams timeshare instead of
overlapping; the small-chunk penalty is small-M GEMM inefficiency.  No
production pipeline path; the serial loop at CHUNK_MAX_GB=2 stands.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 08:27:19 +00:00
..