chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[Ascend] support Kimi-K2-Thinking (#12759)

Browse Source 
Co-authored-by: ZhengdQin <zhengdqin@gmail.com>
Co-authored-by: richhuan <huan_rz@qq.com>
Co-authored-by: ZhengdQin <46387172+ZhengdQin@users.noreply.github.com>
This commit is contained in:
Yijie Zhu
2025-11-22 17:33:07 +08:00
committed by GitHub
co-authored by ZhengdQin richhuan ZhengdQin
parent 0eea17e3b5
commit a92afb00c6
4 changed files with 551 additions and 172 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/sglang/srt/model_executor/model_runner.py
+1 -1
View File
@@ -217,7 +217,7 @@ _is_xpu_xmx_available = xpu_has_xmx_support()
SGLANG_CI_SMALL_KV_SIZE = os.getenv("SGLANG_CI_SMALL_KV_SIZE", None)
# Detect stragger ranks in model loading
UNBALANCED_MODEL_LOADING_TIMEOUT_S = 300
UNBALANCED_MODEL_LOADING_TIMEOUT_S = 480 # leave more time for post data processing
# the ratio of mamba cache pool size to max_running_requests, it will be safe when it is larger than 2 (yizhang2077)
MAMBA_CACHE_SIZE_MAX_RUNNING_REQUESTS_RATIO = 3