Files
sglang/python
leavelet c1637cfb68 B300 NextN fp4: review cleanups — keep else-slice, non-routed draft trtllm, weight_scale_2 probe
Adversarial review of b3a4f4174f found 3 items (Bug B 2b8593ff24 accepted as-is):
1) Re-add the modelopt_quant else-branch input-scale EP-slice (reverting it re-opened a general latent
   EP>1 NVFP4 crash on the plain/triton path: --moe-runner-backend triton --quantization modelopt_fp4
   --ep>1). No-op at ep==1; orthogonal defense-in-depth (draft now takes the trtllm scalar branch).
2) Draft uses the NON-routed flashinfer_trtllm (FlashInferFP4MoE only does the non-routed kernel; the
   _routed impl-class isn't wired). Reject explicit --speculative-moe-runner-backend flashinfer_trtllm_routed.
3) Tighten the checkpoint probe to key on weight_scale_2 (fp4-specific) instead of the generic
   weight_scale substring (which also matches fp8 weight_scale_inv).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 17:48:06 +00:00
..
2026-06-10 05:54:43 +08:00