chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

fix(config): restore GlmMoeDsa index_topk_freq dropped by HF config load

Browse Source 
GlmMoeDsaConfig.__init__ has no **kwargs, so transformers drops the raw
config.json fields the IndexCache/DSA path reads via getattr (index_topk_freq,
index_topk_pattern, index_skip_topk_offset) and may clobber qk_rope_head_dim —
only named fields like index_topk survive. Re-read them from config.json and
restore in get_config for GlmMoeDsaForCausalLM, matching upstream #27114.

Without this, setting index_topk_freq in the model config has NO effect
(getattr falls back to 1 -> IndexCache stays off). Verified on the B300 image:
get_config("/ssd/models/GLM-5.1-FP8") now returns index_topk_freq=4.

Refs: WI-2026-06-07-001

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 92ff8b47470c61462e252bc7ba428203ff3c324c)
This commit is contained in:
孙远航
2026-06-07 19:32:26 +00:00
committed by laoyao0822
parent ce3b5bc874
commit 98fdae3375
1 changed files with 21 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

21
python/sglang/srt/utils/hf_transformers_utils.py
View File

@@ -543,6 +543,27 @@ def get_config(
else:
raise
if (
config.architectures is not None
and config.architectures[0] == "GlmMoeDsaForCausalLM"
):
# GlmMoeDsaConfig.__init__ has no **kwargs, so HF drops raw config.json
# fields the DSA / IndexCache path reads via getattr (index_topk_freq,
# index_topk_pattern, index_skip_topk_offset) and may clobber
# qk_rope_head_dim. Re-read them from config.json and restore.
# (upstream PR #27114) Remove once transformers >= 5.10 (HF PR #46338).
raw_config, _ = PretrainedConfig.get_config_dict(model, revision=revision)
for key in (
"qk_rope_head_dim",
"index_topk_freq",
"index_topk_pattern",
"index_skip_topk_offset",
):
if key in raw_config:
setattr(config, key, raw_config[key])
if hasattr(config, "qk_head_dim") and hasattr(config, "qk_nope_head_dim"):
config.qk_head_dim = config.qk_nope_head_dim + config.qk_rope_head_dim
if (
config.architectures is not None
and config.architectures[0] == "Phi4MMForCausalLM"