chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Tiny env cleanup in deepgemm (#15706)

Browse Source 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
This commit is contained in:
vincentzed
2025-12-24 03:58:46 -05:00
committed by GitHub
parent f65fa04748
commit ddb3970eb4
4 changed files with 7 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
python/sglang/compile_deep_gemm.py
View File

@@ -33,7 +33,7 @@ envs.SGLANG_IN_DEEPGEMM_PRECOMPILE_STAGE.set(True)
# Force enable deep gemm
envs.SGLANG_ENABLE_JIT_DEEPGEMM.set(True)
# Force enable mha chunked kv for DeepSeek V3 to avoid missing kv_b_proj DeepGEMM case
os.environ["SGLANG_CHUNKED_PREFIX_CACHE_THRESHOLD"] = "0"
envs.SGLANG_CHUNKED_PREFIX_CACHE_THRESHOLD.set(0)
@dataclasses.dataclass