[model-gateway] Add PostgreSQL support to binding (#13766)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Simo Lin <linsimo.mark@gmail.com> Co-authored-by: Chang Su <chang.s.su@oracle.com>
This commit is contained in:
@@ -275,6 +275,7 @@ PD deployments can specify `--prefill-selector` and `--decode-selector` plus the
|
||||
| `memory` (default) | In-memory storage for quick prototyping. | `--history-backend memory` |
|
||||
| `none` | No persistence; APIs operate but store nothing. | `--history-backend none` |
|
||||
| `oracle` | Oracle Autonomous Database-backed storage (pooled connections). | `--history-backend oracle` |
|
||||
| `postgres` | PostgreSQL Database-backed storage (pooled connections). | `--history-backend postgres` |
|
||||
|
||||
Oracle configuration (choose DSN *or* TNS alias):
|
||||
Install the Oracle Instant Client and set `LD_LIBRARY_PATH` accordingly.
|
||||
|
||||
@@ -454,6 +454,7 @@ Public health endpoints (`/liveness`, `/readiness`, `/health`, `/health_generate
|
||||
- `--history-backend memory` (default) stores responses and conversations in-process.
|
||||
- `--history-backend none` disables persistence while keeping APIs.
|
||||
- `--history-backend oracle` uses Oracle Autonomous Database; provide credentials via flags or environment variables.
|
||||
- `--history-backend postgres` uses PostgreSQL Database.
|
||||
- Conversation item storage mirrors the history backend (Oracle or memory). The same storage powers OpenAI `/responses` and conversation APIs.
|
||||
|
||||
### History Backend (OpenAI Router Mode)
|
||||
@@ -465,6 +466,7 @@ Store conversation and response data for tracking, debugging, or analytics.
|
||||
- **Memory** (default): In-memory storage, fast but ephemeral.
|
||||
- **None**: No storage, minimal overhead.
|
||||
- **Oracle**: Persistent storage backed by Oracle Autonomous Database.
|
||||
- **Postgres**: Persistent storage backed by PostgreSQL Database.
|
||||
|
||||
```bash
|
||||
# Memory backend (default)
|
||||
@@ -484,6 +486,12 @@ python3 -m sglang_router.launch_router \
|
||||
--backend openai \
|
||||
--worker-urls https://api.openai.com \
|
||||
--history-backend oracle
|
||||
|
||||
# PostgreSQL backend
|
||||
python3 -m sglang_router.launch_router \
|
||||
--backend openai \
|
||||
--worker-urls https://api.openai.com \
|
||||
--history-backend postgres
|
||||
```
|
||||
|
||||
#### Oracle configuration
|
||||
|
||||
@@ -210,6 +210,8 @@ class Router:
|
||||
"oracle_pool_min",
|
||||
"oracle_pool_max",
|
||||
"oracle_pool_timeout_secs",
|
||||
"postgres_db_url",
|
||||
"postgres_pool_max",
|
||||
]
|
||||
for field in fields_to_remove:
|
||||
args_dict.pop(field, None)
|
||||
|
||||
@@ -109,6 +109,8 @@ class RouterArgs:
|
||||
oracle_pool_min: int = 1
|
||||
oracle_pool_max: int = 16
|
||||
oracle_pool_timeout_secs: int = 30
|
||||
postgres_db_url: Optional[str] = None
|
||||
postgres_pool_max: int = 16
|
||||
# mTLS configuration for worker communication
|
||||
client_cert_path: Optional[str] = None
|
||||
client_key_path: Optional[str] = None
|
||||
@@ -598,6 +600,19 @@ class RouterArgs:
|
||||
),
|
||||
help="Oracle connection pool timeout in seconds (default: 30, env: ATP_POOL_TIMEOUT_SECS)",
|
||||
)
|
||||
# Postgres configuration
|
||||
parser.add_argument(
|
||||
f"--{prefix}postgres-db-url",
|
||||
type=str,
|
||||
default=os.getenv("POSTGRES_DB_URL"),
|
||||
help="PostgreSQL database connection URL (env: POSTGRES_DB_URL)",
|
||||
)
|
||||
parser.add_argument(
|
||||
f"--{prefix}postgres-pool-max",
|
||||
type=int,
|
||||
default=int(os.getenv("POSTGRES_POOL_MAX", RouterArgs.postgres_pool_max)),
|
||||
help="Maximum PostgreSQL connection pool size (default: 16, env: POSTGRES_POOL_MAX)",
|
||||
)
|
||||
# mTLS configuration
|
||||
parser.add_argument(
|
||||
f"--{prefix}client-cert-path",
|
||||
|
||||
Reference in New Issue
Block a user