chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

feat: limit peak memory usage when computing logprobs (#6318)

Browse Source 
Signed-off-by: Zhao Chen <zhaochen.zju@gmail.com>
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
This commit is contained in:
Zhao Chen
2025-11-04 15:53:20 +08:00
committed by GitHub
parent fef3a6b63b
commit d5fa019c36
3 changed files with 651 additions and 189 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
python/sglang/srt/environ.py
View File

@@ -273,6 +273,10 @@ class Envs:
# Sparse Embeddings
SGLANG_EMBEDDINGS_SPARSE_HEAD = EnvStr(None)
# Logits processor
SGLANG_ENABLE_LOGITS_PROCESSER_CHUNK = EnvBool(False)
SGLANG_LOGITS_PROCESSER_CHUNK_SIZE = EnvInt(2048)
# Tool-Call behavior
SGLANG_TOOL_STRICT_LEVEL = EnvInt(ToolStrictLevel.OFF)