feat(megamoe): add fp4 group16 to group32 requant path
Add a utility and synthetic benchmark path for converting FP4 group16 tensors into group32 tensors that the existing SM100 block32 MegaMoE kernels can consume. Document that this is a requantization path rather than a lossless metadata rewrite, so GLM-5.2 accuracy validation is still required before production use. Tested: PYTHONPYCACHEPREFIX=/private/tmp/deepgemm_pycache python3 -m py_compile deep_gemm/utils/math.py tests/test_layout.py tests/test_mega_moe.py Tested: git diff --check Not-tested: CUDA build, SM100/B300 runtime, and GLM-5.2 accuracy validation are not available locally.
This commit is contained in:
@@ -15,6 +15,9 @@ carried an implicit group32 assumption.
|
||||
scale transpose with a group16-aware 128-element tiling.
|
||||
- `tests/test_mega_moe.py` exposes `--weight-gran-k 16|32` so synthetic runs can
|
||||
reproduce GLM-style group16 inputs without loading model weights.
|
||||
- `requant_fp4_to_gran_k(...)` provides an explicit experiment path for
|
||||
converting a group16 FP4 checkpoint tensor into a group32 tensor that the
|
||||
existing block32 kernels can consume.
|
||||
- The fused SM100 MegaMoE compute API now performs an explicit capability check
|
||||
for `recipe=(1, 1, 16)` instead of failing earlier with
|
||||
`Unknown SF transformation`.
|
||||
@@ -33,6 +36,19 @@ group16 correctly requires auditing at least:
|
||||
Until that kernel work is complete and validated on B300/SM100, group16 should
|
||||
be treated as layout-supported but fused-compute unsupported.
|
||||
|
||||
## Requantization path
|
||||
|
||||
If the SM100 MXF4 MMA path cannot consume group16 scales directly, the fallback
|
||||
engineering path is checkpoint conversion:
|
||||
|
||||
1. dequantize group16 FP4 values with their original scales;
|
||||
2. requantize the restored values with group32 scales;
|
||||
3. run the existing `recipe=(1, 1, 32)` MegaMoE kernel.
|
||||
|
||||
This is not a lossless metadata rewrite. It changes the quantized checkpoint and
|
||||
must go through GLM-5.2 accuracy validation before it can be used as a production
|
||||
answer.
|
||||
|
||||
## Validation target
|
||||
|
||||
After kernel support is added, validate with:
|
||||
@@ -41,4 +57,6 @@ After kernel support is added, validate with:
|
||||
- `tests/test_layout.py` on SM100 for `gran_k=16`;
|
||||
- `tests/test_mega_moe.py --weight-gran-k 16 --ncu-profile-only` for synthetic
|
||||
fused execution;
|
||||
- `tests/test_mega_moe.py --weight-gran-k 16 --requant-group16-to-group32` for
|
||||
the checkpoint conversion path against existing block32 kernels;
|
||||
- SGLang GLM-5.2 NVFP4 real-weight layout build and 8-card e2e smoke.
|
||||
|
||||
Reference in New Issue
Block a user