Revert "tiny remove deprecated endpoint call" (#14533)
This commit is contained in:
@@ -45,7 +45,7 @@ SGLang Model Gateway is a high-performance model-routing gateway for large-scale
|
||||
## Architecture
|
||||
|
||||
### Control Plane
|
||||
- **Worker Manager** discovers capabilities (`/server_info`, `/model_info`), tracks load, and registers/removes workers in the shared registry.
|
||||
- **Worker Manager** discovers capabilities (`/get_server_info`, `/get_model_info`), tracks load, and registers/removes workers in the shared registry.
|
||||
- **Job Queue** serializes add/remove requests and exposes status (`/workers/{url}`) so clients can track onboarding progress.
|
||||
- **Load Monitor** feeds cache-aware and power-of-two policies with live worker load statistics.
|
||||
- **Health Checker** continuously probes workers and updates readiness, circuit breaker state, and router metrics.
|
||||
|
||||
@@ -219,7 +219,7 @@ Please consult the documentation below and [server_args.py](https://github.com/s
|
||||
| Argument | Description | Defaults | Options |
|
||||
| --- | --- | --- | --- |
|
||||
| `--json-model-override-args` | A dictionary in JSON string format used to override default model configurations. | `{}` | Type: str |
|
||||
| `--preferred-sampling-params` | json-formatted sampling settings that will be returned in /model_info | `None` | Type: str |
|
||||
| `--preferred-sampling-params` | json-formatted sampling settings that will be returned in /get_model_info | `None` | Type: str |
|
||||
|
||||
## LoRA
|
||||
| Argument | Description | Defaults | Options |
|
||||
|
||||
@@ -9,8 +9,8 @@
|
||||
"Apart from the OpenAI compatible APIs, the SGLang Runtime also provides its native server APIs. We introduce the following APIs:\n",
|
||||
"\n",
|
||||
"- `/generate` (text generation model)\n",
|
||||
"- `/model_info`\n",
|
||||
"- `/server_info`\n",
|
||||
"- `/get_model_info`\n",
|
||||
"- `/get_server_info`\n",
|
||||
"- `/health`\n",
|
||||
"- `/health_generate`\n",
|
||||
"- `/flush_cache`\n",
|
||||
@@ -99,7 +99,7 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"url = f\"http://localhost:{port}/model_info\"\n",
|
||||
"url = f\"http://localhost:{port}/get_model_info\"\n",
|
||||
"\n",
|
||||
"response = requests.get(url)\n",
|
||||
"response_json = response.json()\n",
|
||||
@@ -127,7 +127,7 @@
|
||||
"source": [
|
||||
"## Get Server Info\n",
|
||||
"Gets the server information including CLI arguments, token limits, and memory pool sizes.\n",
|
||||
"- Note: `server_info` merges the following deprecated endpoints:\n",
|
||||
"- Note: `get_server_info` merges the following deprecated endpoints:\n",
|
||||
" - `get_server_args`\n",
|
||||
" - `get_memory_pool_size` \n",
|
||||
" - `get_max_total_num_tokens`"
|
||||
@@ -139,7 +139,7 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"url = f\"http://localhost:{port}/server_info\"\n",
|
||||
"url = f\"http://localhost:{port}/get_server_info\"\n",
|
||||
"\n",
|
||||
"response = requests.get(url)\n",
|
||||
"print_highlight(response.text)"
|
||||
|
||||
@@ -352,4 +352,4 @@ python3 -m sglang.bench_serving \
|
||||
### Notes
|
||||
|
||||
- The script raises the file descriptor soft limit (`RLIMIT_NOFILE`) to help with many concurrent connections.
|
||||
- For sglang, `/server_info` is queried post-run to report speculative decoding accept length when available.
|
||||
- For sglang, `/get_server_info` is queried post-run to report speculative decoding accept length when available.
|
||||
|
||||
Reference in New Issue
Block a user