[model-gateway] Add PostgreSQL support to binding (#13766)

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
Co-authored-by: Chang Su <chang.s.su@oracle.com>
This commit is contained in:
Wenyi Xu
2025-11-27 01:11:13 +08:00
committed by GitHub
parent 9dab534b35
commit 5e70880e64
4 changed files with 26 additions and 0 deletions

View File

@@ -275,6 +275,7 @@ PD deployments can specify `--prefill-selector` and `--decode-selector` plus the
| `memory` (default) | In-memory storage for quick prototyping. | `--history-backend memory` |
| `none` | No persistence; APIs operate but store nothing. | `--history-backend none` |
| `oracle` | Oracle Autonomous Database-backed storage (pooled connections). | `--history-backend oracle` |
| `postgres` | PostgreSQL Database-backed storage (pooled connections). | `--history-backend postgres` |
Oracle configuration (choose DSN *or* TNS alias):
Install the Oracle Instant Client and set `LD_LIBRARY_PATH` accordingly.

View File

@@ -454,6 +454,7 @@ Public health endpoints (`/liveness`, `/readiness`, `/health`, `/health_generate
- `--history-backend memory` (default) stores responses and conversations in-process.
- `--history-backend none` disables persistence while keeping APIs.
- `--history-backend oracle` uses Oracle Autonomous Database; provide credentials via flags or environment variables.
- `--history-backend postgres` uses PostgreSQL Database.
- Conversation item storage mirrors the history backend (Oracle or memory). The same storage powers OpenAI `/responses` and conversation APIs.
### History Backend (OpenAI Router Mode)
@@ -465,6 +466,7 @@ Store conversation and response data for tracking, debugging, or analytics.
- **Memory** (default): In-memory storage, fast but ephemeral.
- **None**: No storage, minimal overhead.
- **Oracle**: Persistent storage backed by Oracle Autonomous Database.
- **Postgres**: Persistent storage backed by PostgreSQL Database.
```bash
# Memory backend (default)
@@ -484,6 +486,12 @@ python3 -m sglang_router.launch_router \
--backend openai \
--worker-urls https://api.openai.com \
--history-backend oracle
# PostgreSQL backend
python3 -m sglang_router.launch_router \
--backend openai \
--worker-urls https://api.openai.com \
--history-backend postgres
```
#### Oracle configuration

View File

@@ -210,6 +210,8 @@ class Router:
"oracle_pool_min",
"oracle_pool_max",
"oracle_pool_timeout_secs",
"postgres_db_url",
"postgres_pool_max",
]
for field in fields_to_remove:
args_dict.pop(field, None)

View File

@@ -109,6 +109,8 @@ class RouterArgs:
oracle_pool_min: int = 1
oracle_pool_max: int = 16
oracle_pool_timeout_secs: int = 30
postgres_db_url: Optional[str] = None
postgres_pool_max: int = 16
# mTLS configuration for worker communication
client_cert_path: Optional[str] = None
client_key_path: Optional[str] = None
@@ -598,6 +600,19 @@ class RouterArgs:
),
help="Oracle connection pool timeout in seconds (default: 30, env: ATP_POOL_TIMEOUT_SECS)",
)
# Postgres configuration
parser.add_argument(
f"--{prefix}postgres-db-url",
type=str,
default=os.getenv("POSTGRES_DB_URL"),
help="PostgreSQL database connection URL (env: POSTGRES_DB_URL)",
)
parser.add_argument(
f"--{prefix}postgres-pool-max",
type=int,
default=int(os.getenv("POSTGRES_POOL_MAX", RouterArgs.postgres_pool_max)),
help="Maximum PostgreSQL connection pool size (default: 16, env: POSTGRES_POOL_MAX)",
)
# mTLS configuration
parser.add_argument(
f"--{prefix}client-cert-path",