chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 13 Packages Projects Releases Wiki Activity

feat: add forward timeout (#17831)

Browse Source 
Co-authored-by: qiuxuan.lzw <qiuxuan.lzw@alibaba-inc.com>
This commit is contained in:
StonyPort
2026-01-30 08:52:29 +08:00
committed by GitHub
co-authored by qiuxuan.lzw
parent 6a6b36367e
commit 2b3408ff14
4 changed files with 63 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/references/environment_variables.md
+2 -1
View File
@@ -17,7 +17,8 @@ SGLang supports various environment variables that can be used to configure its
| `SGLANG_HEALTH_CHECK_TIMEOUT` | Timeout for health check in seconds | `20` |
| `SGLANG_EPLB_HEATMAP_COLLECTION_INTERVAL` | The interval of passes to collect the metric of selected count of physical experts on each layer and GPU rank. 0 means disabled. | `0` |
| `SGLANG_FORWARD_UNKNOWN_TOOLS` | Forward unknown tool calls to clients instead of dropping them | `false` (drop unknown tools) |
| `SGLANG_QUEUED_TIMEOUT_MS` | Timeout (in ms) for requests in the waiting queue | `-1` |
| `SGLANG_QUEUED_TIMEOUT_MS` | Timeout (in ms) for requests in the waiting queue | `-1` |
| `SGLANG_FORWARD_TIMEOUT_MS` | Timeout (in ms) for requests in the forward batch | `-1` |
## Performance Tuning