[model-gateway] Add Redis support as a history backend (#16300)

This commit is contained in:
Wenyi Xu
2026-01-11 01:03:00 -08:00
committed by GitHub
parent 7b089ae4e0
commit 3c16c58619
12 changed files with 1215 additions and 10 deletions
@@ -858,6 +858,7 @@ Prefill pods can expose bootstrap ports via the `sglang.ai/bootstrap-port` annot
| `none` | No persistence | `--history-backend none` |
| `oracle` | Oracle Autonomous Database | `--history-backend oracle` |
| `postgres` | PostgreSQL Database | `--history-backend postgres` |
| `redis` | Redis | `--history-backend redis` |
### Oracle Configuration
@@ -892,6 +893,22 @@ python -m sglang_router.launch_router \
--history-backend postgres
```
### Redis Configuration
```bash
export REDIS_URL="redis://localhost:6379"
export REDIS_POOL_MAX=16
export REDIS_RETENTION_DAYS=30
python -m sglang_router.launch_router \
--backend openai \
--worker-urls https://api.openai.com \
--history-backend redis \
--redis-retention-days 30
```
Use `--redis-retention-days -1` for persistent storage (default is 30 days).
---
## WASM Middleware