ROOT PROBLEM (opus-diagnosed): the NextN/MTP draft MoE is a normal DeepSeek-V3 fp4 MoE, identical in
kind to the main model's — but our fork forced the speculative MoE backend to "auto" (+ a blanket
assert forbidding trtllm-as-speculative), so the draft built under speculative_moe_backend_context →
MOE_RUNNER_BACKEND=AUTO → plain FusedMoE → the non-flashinfer process_weights 'else' branch that
doesn't reconcile EP scale sharding → w13_input_scale(global 256) × w13_weight_scale_2(local 64) crash.
The trtllm-as-speculative ban is over-broad: the Renormalize-only routing constraint is for the BF16
trtllm MoE; the FP4 trtllm kernel supports the DeepSeek routing the NextN carries (FlashInferFP4MoE.
forward_impl). So the elegant fix is to let the draft run the SAME flashinfer_trtllm path as main when
the NextN is fp4 — scoped via the checkpoint's safetensors index (ground truth: do the MTP experts
ship fp4 scales). DeepSeek-R1/V3-FP4 ship a bf16 MTP -> still drop to bf16 + keep the non-trtllm path.
- weight_utils: add shared nextn_moe_is_fp4_quantized_in_checkpoint + load_safetensors_index_weight_map.
- deepseek_nextn: delegate the keep/drop probe to the shared helper (one source of truth).
- server_args._handle_speculative_decoding: for an fp4 NextN, default the speculative backend to the
main trtllm backend (instead of "auto"); narrow the assert to allow trtllm only for an fp4 MTP.
- modelopt_quant: REVERT the else-branch input-scale slice band-aid (de726b4e7e) — no longer hit; the
fp4 draft now takes the trtllm .max() scalar branch, same as main.
Bug B (k_rope 256-vs-64 in the trtllm NSA *prefill* path under CP) is independent and still pending.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
leavelet
b3a4f4174f