chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
11,255 Commits 1 Branch 0 Tags
89ba17ad056d36aa6e2e4f66176e873fcb12734a
Commit Graph
4 Commits
Author SHA1 Message Date
LuminolT 89ba17ad05 feat(moe): add fp8 megamoe fused path and nvfp4 layout guards 
Wire MegaMoE FP8 through DeepGEMM's fused fp8_mega_moe path, preserve fallback runner layouts, and add explicit NVFP4 group-size guardrails for unsupported DeepGEMM scale transforms.

Tested: PYTHONPYCACHEPREFIX=/private/tmp/sglang_pycache python3 -m py_compile python/sglang/srt/layers/moe/mega_moe.py python/sglang/srt/layers/moe/moe_runner/flashinfer_trtllm.py python/sglang/srt/layers/quantization/fp8.py python/sglang/srt/layers/quantization/modelopt_quant.py test/registered/unit/moe/test_glm_megamoe.py

Tested: git diff --check

Not-tested: python3 -m unittest test/registered/unit/moe/test_glm_megamoe.py (local Python environments do not have torch installed).
 2026-07-08 11:42:51 +08:00
LuminolT eb7f44a8ee fix(moe): pass megamoe fp4 weights as int8 to deepgemm 
DeepGEMM's TVM FFI binding rejects PyTorch FP4x2 DLPack lanes, while the MegaMoE kernel expects the same packed bytes. Convert FP4 packed expert weights to int8 views immediately before fp8_fp4_mega_moe without changing storage or scale tensors.

Tested: python3 -m pytest -q test/registered/unit/moe/test_glm_megamoe.py on remote B300 container (5 passed).

Tested: 1x B300 synthetic forward_mega_moe smoke passed.

Tested: 8x B300 synthetic direct DeepGEMM MegaMoE and forward_mega_moe wrapper smoke passed.
 2026-07-07 14:31:03 +08:00
LuminolT c00630088c fix(moe): infer megamoe fp4 weight scale group 
Infer the FP4 weight scale group size from the loaded scale tensor instead of hard-coding K/32.

This keeps upstream-style FP4 expert layouts working while allowing GLM/ModelOpt NVFP4 layouts that use K/16 scale columns to build MegaMoE sidecar weights.

Constraint: preserve the existing runner layout and only change MegaMoE sidecar metadata/recipe.

Feature-flag: --moe-a2a-backend=megamoe.

Conflict-hotspots: python/sglang/srt/layers/moe/mega_moe.py.

Scope-risk: actual DeepGEMM recipe support still needs target GPU runtime validation.

Tested: PYTHONPYCACHEPREFIX=/private/tmp/sglang_pycache python3 -m py_compile python/sglang/srt/layers/moe/mega_moe.py test/registered/unit/moe/test_glm_megamoe.py.

Tested: git diff --check.

Not-tested: GLM 5.2 MegaMoE GPU e2e; local environment lacks target runtime and hardware.
 2026-07-06 10:46:18 +08:00
LuminolT 93e3840578 test(moe): add GLM MegaMoE validation coverage 
Add parser coverage for --moe-a2a-backend=megamoe and local unit coverage for MegaMoE gating, GLM fast-path routing, fallback routing, and sidecar weight preservation.

Use monkeypatch-style tests so the forward and weight-layout checks do not require loading GLM 5.2 or importing a real deep_gemm runtime.

Constraint: tests must not stage or depend on local skill/doc artifacts.

Feature-flag: --moe-a2a-backend=megamoe.

Conflict-hotspots: test/registered/unit/server_args/test_server_args.py, test/registered/unit/moe/test_glm_megamoe.py.

Scope-risk: runtime GPU e2e still needs target Blackwell/DeepGEMM environment.

Tested: PYTHONPYCACHEPREFIX=/private/tmp/sglang_pycache python3 -m py_compile test/registered/unit/server_args/test_server_args.py test/registered/unit/moe/test_glm_megamoe.py.

Tested: git diff --check.

Not-tested: PYTHONPATH=python python3 -m pytest test/registered/unit/moe/test_glm_megamoe.py -q; local Python has no pytest.

Not-tested: PYTHONPATH=python python3 -m unittest test.registered.unit.moe.test_glm_megamoe -v; local Python has no torch.

Not-tested: GLM 5.2 MegaMoE GPU e2e; local environment lacks target runtime and hardware.
 2026-07-06 10:44:05 +08:00