Revert "tiny remove deprecated endpoint call" (#14533)

This commit is contained in:
Baizhou Zhang
2025-12-05 23:48:54 -08:00
committed by GitHub
parent d257bf87b9
commit 42fcf5438f
36 changed files with 59 additions and 59 deletions

View File

@@ -45,7 +45,7 @@ SGLang Model Gateway is a high-performance model-routing gateway for large-scale
## Architecture
### Control Plane
- **Worker Manager** discovers capabilities (`/server_info`, `/model_info`), tracks load, and registers/removes workers in the shared registry.
- **Worker Manager** discovers capabilities (`/get_server_info`, `/get_model_info`), tracks load, and registers/removes workers in the shared registry.
- **Job Queue** serializes add/remove requests and exposes status (`/workers/{url}`) so clients can track onboarding progress.
- **Load Monitor** feeds cache-aware and power-of-two policies with live worker load statistics.
- **Health Checker** continuously probes workers and updates readiness, circuit breaker state, and router metrics.

View File

@@ -219,7 +219,7 @@ Please consult the documentation below and [server_args.py](https://github.com/s
| Argument | Description | Defaults | Options |
| --- | --- | --- | --- |
| `--json-model-override-args` | A dictionary in JSON string format used to override default model configurations. | `{}` | Type: str |
| `--preferred-sampling-params` | json-formatted sampling settings that will be returned in /model_info | `None` | Type: str |
| `--preferred-sampling-params` | json-formatted sampling settings that will be returned in /get_model_info | `None` | Type: str |
## LoRA
| Argument | Description | Defaults | Options |

View File

@@ -9,8 +9,8 @@
"Apart from the OpenAI compatible APIs, the SGLang Runtime also provides its native server APIs. We introduce the following APIs:\n",
"\n",
"- `/generate` (text generation model)\n",
"- `/model_info`\n",
"- `/server_info`\n",
"- `/get_model_info`\n",
"- `/get_server_info`\n",
"- `/health`\n",
"- `/health_generate`\n",
"- `/flush_cache`\n",
@@ -99,7 +99,7 @@
"metadata": {},
"outputs": [],
"source": [
"url = f\"http://localhost:{port}/model_info\"\n",
"url = f\"http://localhost:{port}/get_model_info\"\n",
"\n",
"response = requests.get(url)\n",
"response_json = response.json()\n",
@@ -127,7 +127,7 @@
"source": [
"## Get Server Info\n",
"Gets the server information including CLI arguments, token limits, and memory pool sizes.\n",
"- Note: `server_info` merges the following deprecated endpoints:\n",
"- Note: `get_server_info` merges the following deprecated endpoints:\n",
" - `get_server_args`\n",
" - `get_memory_pool_size` \n",
" - `get_max_total_num_tokens`"
@@ -139,7 +139,7 @@
"metadata": {},
"outputs": [],
"source": [
"url = f\"http://localhost:{port}/server_info\"\n",
"url = f\"http://localhost:{port}/get_server_info\"\n",
"\n",
"response = requests.get(url)\n",
"print_highlight(response.text)"

View File

@@ -352,4 +352,4 @@ python3 -m sglang.bench_serving \
### Notes
- The script raises the file descriptor soft limit (`RLIMIT_NOFILE`) to help with many concurrent connections.
- For sglang, `/server_info` is queried post-run to report speculative decoding accept length when available.
- For sglang, `/get_server_info` is queried post-run to report speculative decoding accept length when available.