[model-gateway] Add Redis support as a history backend (#16300)
This commit is contained in:
@@ -858,6 +858,7 @@ Prefill pods can expose bootstrap ports via the `sglang.ai/bootstrap-port` annot
|
||||
| `none` | No persistence | `--history-backend none` |
|
||||
| `oracle` | Oracle Autonomous Database | `--history-backend oracle` |
|
||||
| `postgres` | PostgreSQL Database | `--history-backend postgres` |
|
||||
| `redis` | Redis | `--history-backend redis` |
|
||||
|
||||
### Oracle Configuration
|
||||
|
||||
@@ -892,6 +893,22 @@ python -m sglang_router.launch_router \
|
||||
--history-backend postgres
|
||||
```
|
||||
|
||||
### Redis Configuration
|
||||
|
||||
```bash
|
||||
export REDIS_URL="redis://localhost:6379"
|
||||
export REDIS_POOL_MAX=16
|
||||
export REDIS_RETENTION_DAYS=30
|
||||
|
||||
python -m sglang_router.launch_router \
|
||||
--backend openai \
|
||||
--worker-urls https://api.openai.com \
|
||||
--history-backend redis \
|
||||
--redis-retention-days 30
|
||||
```
|
||||
|
||||
Use `--redis-retention-days -1` for persistent storage (default is 30 days).
|
||||
|
||||
---
|
||||
|
||||
## WASM Middleware
|
||||
|
||||
Reference in New Issue
Block a user