B300 NVFP4: fix FlashInferFP4MoE.forward attr names to match HEAD weight-prep

The team-custom FlashInferFP4MoE.forward_impl read gemm{1,2}_{weights,scales}_fp4_shuffled, but the
HEAD-advanced align_fp4_moe_weights_for_flashinfer_trtllm now stores into w13_weight/w2_weight/
w13_weight_scale/w2_weight_scale (weights uint8, scales already fp8). Rename the 4 reads (Option a:
preserves the team forward incl. its DeepSeekV3 fp32 router_logits cast — GLM-5.2 uses DeepSeekV3
routing, so option-b delegation that drops the cast was avoided). g1_scale_c/g1_alphas/g2_alphas/
w13_input_scale_quant already match HEAD writes (opus-verified 1:1 format/semantics, no silent-garbage).
Cannot wholesale-port layer.py/ep_moe (team EP/CP infra). Watch at launch: draft accept-len + sane output.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-21 16:24:17 +00:00
parent ae5e98cf86
commit 5e670760bd

View File

@@ -1376,18 +1376,18 @@ class FlashInferFP4MoE(FusedMoE):
hidden_states_scale=hs_scale_linear.view(torch.float8_e4m3fn).reshape(
*hs_scale_linear.shape[:-1], -1
),
gemm1_weights=self.gemm1_weights_fp4_shuffled.data,
gemm1_weights_scale=self.gemm1_scales_fp4_shuffled.data.view(
torch.float8_e4m3fn
),
# B300 port: HEAD's align_fp4_moe_weights_for_flashinfer_trtllm now
# stores the prepared (shuffled) fp4 weights/scales into the standard
# param names instead of gemm*_fp4_shuffled (weights=uint8, scales
# already fp8_e4m3fn — the .view below is a no-op kept for clarity).
gemm1_weights=self.w13_weight.data,
gemm1_weights_scale=self.w13_weight_scale.data.view(torch.float8_e4m3fn),
gemm1_bias=None,
gemm1_alpha=None,
gemm1_beta=None,
gemm1_clamp_limit=None,
gemm2_weights=self.gemm2_weights_fp4_shuffled.data,
gemm2_weights_scale=self.gemm2_scales_fp4_shuffled.data.view(
torch.float8_e4m3fn
),
gemm2_weights=self.w2_weight.data,
gemm2_weights_scale=self.w2_weight_scale.data.view(torch.float8_e4m3fn),
gemm2_bias=None,
output1_scale_scalar=self.g1_scale_c.data,
output1_scale_gate_scalar=self.g1_alphas.data,