chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

feat: add request queued timeout (#17143)

Browse Source 
Co-authored-by: qiuxuan.lzw <qiuxuan.lzw@alibaba-inc.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
This commit is contained in:
StonyPort
2026-01-16 17:55:09 +08:00
committed by GitHub
parent a1dd3d48ac
commit 3355b6e21b
5 changed files with 78 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/references/environment_variables.md
View File

@@ -16,6 +16,7 @@ SGLang supports various environment variables that can be used to configure its
| `SGLANG_HEALTH_CHECK_TIMEOUT` | Timeout for health check in seconds | `20` |
| `SGLANG_EPLB_HEATMAP_COLLECTION_INTERVAL` | The interval of passes to collect the metric of selected count of physical experts on each layer and GPU rank. 0 means disabled. | `0` |
| `SGLANG_FORWARD_UNKNOWN_TOOLS` | Forward unknown tool calls to clients instead of dropping them | `false` (drop unknown tools) |
| `SGLANG_QUEUED_TIMEOUT_MS` | Timeout (in ms) for requests in the waiting queue | `-1` |
## Performance Tuning