The bench gains a pipelined mode (topk_N event-gated on a side stream,
overlapping logits_{N+1}). Byte-equality validation was dropped after
establishing the kernel chain is run-to-run nondeterministic even
serial-vs-serial (near-equal fp32 selection).
Result on g0033 H200: pipelining recovers only 0.3-8.9% where small
chunks cost +15-46% — fp8_mqa_logits and fast_topk_transform_fused are
both SM-saturating, so concurrent streams timeshare instead of
overlapping; the small-chunk penalty is small-M GEMM inefficiency. No
production pipeline path; the serial loop at CHUNK_MAX_GB=2 stands.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Measures the real cost of shrinking SGLANG_NSA_MQA_LOGITS_CHUNK_MAX_GB:
the faithful indexer loop (deep_gemm.fp8_mqa_logits +
fast_topk_transform_fused, serial) at GLM-5.1 shapes (H=32, D=128,
topk=2048) across cold-chunk / tail-chunk / warm-continuation /
warm-long scenarios. g0033 1xH200 results: 2GB costs at most +5.5%
(cold 64K chunk) and is -6.7% on the heaviest warm-long shape; the
knee is ~1GB; 0.5GB is +30%. Shrinking 8->2GB frees ~6GB of the
per-batch CP admission budget for KV layer buffers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
leaveletandClaude Fable 5