 Simo Lin
|
3645ed0f73
|
[model-gateway] Add PrefixHash load balancing policy for KV cache-aware routing (#15935)
|
2025-12-27 05:58:05 -05:00 |
|
|
Ratish P
|
ca740a41f3
|
[model-gateway]: fix grpc embedding test (#15934)
|
2025-12-27 05:56:54 -05:00 |
|
|
Simo Lin
|
0e25aa439b
|
[model-gateway] optimize radix tree memory and reduce allocations (#15933)
|
2025-12-27 02:47:30 -08:00 |
|
|
Simo Lin
|
171912a9e3
|
[model-gateway] Add consistent hashing for ManualPolicy routing (#15907)
|
2025-12-26 22:53:03 -08:00 |
|
|
Simo Lin
|
4edee6954a
|
[model-gateway] add JWT/OIDC authentication for control plane APIs (#15850)
|
2025-12-26 18:30:00 -08:00 |
|
|
Simo Lin
|
caa95c7eb4
|
[model-gateway] update ManualPolicy with header-based routing (#15847)
|
2025-12-25 13:04:43 -08:00 |
|
|
fzyzcjy
|
6ef543f99d
|
Use X-SMG-Routing-Key header instead of json body and add tests (#15826)
|
2025-12-25 09:58:14 -08:00 |
|
|
fzyzcjy
|
ccf2330b28
|
Tiny fix missing record_router_upstream_response (#15811)
|
2025-12-25 15:20:43 +08:00 |
|
|
fzyzcjy
|
45adad37d0
|
Add manual routing policy for router (#15586)
|
2025-12-25 11:27:36 +08:00 |
|
|
fzyzcjy
|
1ba897f330
|
Tiny refactor select_workers API for future passing more information (#15596)
|
2025-12-25 10:31:10 +08:00 |
|
|
Simo Lin
|
643aeefe6d
|
[model-gateway] Fix logging module name, parse endpoint context, and tokenizer factory (#15782)
|
2025-12-24 10:54:48 -08:00 |
|
|
Praneth Paruchuri
|
370bd27f3a
|
[model-gateway] Implement Zero-Copy Vision Tensor Access (#15750)
|
2025-12-24 09:15:44 -08:00 |
|
|
Simo Lin
|
2f7c629235
|
[model-gateway] Fix IGW routing and optimize RouterManager (#15741)
|
2025-12-24 11:28:53 -05:00 |
|
|
fzyzcjy
|
d6108166bc
|
Fix smg_http_requests_total semantics (#15655)
|
2025-12-24 17:41:01 +08:00 |
|
|
Arthur Cheng
|
f65fa04748
|
[model-gateway]Enable IGW mode with gRPC router and auto enable IGW when service discovery is turned on (#15459)
|
2025-12-24 00:27:58 -08:00 |
|
|
fzyzcjy
|
99d3bcdfed
|
[model-gateway] add back router worker health metric and fix init state (#15622)
|
2025-12-23 18:50:38 -08:00 |
|
|
fzyzcjy
|
4d64f15086
|
[mode;-gateway] add back fixes of incorrect metrics after worker removal (#15624)
|
2025-12-23 18:50:03 -08:00 |
|
|
Simo Lin
|
846953d9f1
|
[model-gateway] Add tokenize/detokenize HTTP endpoints and tokenizer management (#15702)
|
2025-12-23 17:32:07 -08:00 |
|
|
Simo Lin
|
aa6ac96635
|
[model-gateway] Fix tokenizer caching and improve error handling (#15695)
|
2025-12-23 10:33:38 -08:00 |
|
|
ratish
|
5f3a47d8a7
|
[model-gateway]: add gRPC router embeddings endpoint implementation (#15273)
|
2025-12-23 09:12:22 -08:00 |
|
|
Praneth Paruchuri
|
80ae2229d3
|
[model-gateway] Optimize router selection with lock-free snapshots (#15672)
|
2025-12-23 09:09:30 -08:00 |
|
|
Arthur Cheng
|
dd620987d1
|
[model-gateway] Replace tokenizer with tokenizer registry for dynamic tokenizer loading in gRPC router (#12968)
|
2025-12-23 07:58:47 -08:00 |
|
|
fzyzcjy
|
b736a1525a
|
Tiny add back missing router per attempt response metric (#15621)
|
2025-12-22 09:59:10 -08:00 |
|
|
fzyzcjy
|
6984837d82
|
Fix router gRPC mode launch error caused by async loading (#15368)
|
2025-12-22 09:39:01 -08:00 |
|
|
Simo Lin
|
2142881b7e
|
[model-gateway] return 503 when all workers are circuit-broken (#15611)
|
2025-12-22 04:06:09 -10:00 |
|
|
Simo Lin
|
1167867e8d
|
[model-gateway] add retry support to OpenAI router chat endpoint (#15589)
|
2025-12-21 20:28:51 -08:00 |
|
|
Simo Lin
|
122c250336
|
[model-gateway] add retry and circuit breaker support to gRPC routers (#15585)
|
2025-12-21 17:12:36 -08:00 |
|
|
Simo Lin
|
b9d786059f
|
[model-gateway] refactor WorkerManager with fan_out helper and thin handlers (#15583)
|
2025-12-21 15:29:37 -08:00 |
|
|
Simo Lin
|
1354063a48
|
[model-gateway] add WorkerService abstraction for worker business logic (#15580)
|
2025-12-21 14:35:35 -08:00 |
|
|
Simo Lin
|
e1dcd0df3e
|
[model-gateway] minor code clean up (#15578)
|
2025-12-21 11:00:32 -10:00 |
|
|
shuwenn
|
1d9ba2ce4d
|
[model-gateway] Use UUIDs for router-managed worker resources (#15540)
|
2025-12-21 07:40:03 -10:00 |
|
|
Haoran Wang
|
ef001fb811
|
[model-gateway] /parse/easoning and parse/function_call for sgl-model-gateway (#15568)
|
2025-12-21 07:39:24 -10:00 |
|
|
Haoran Wang
|
26704c23c0
|
[model-gateway]: Tool parser for glm47 (#15520)
|
2025-12-20 18:07:51 -10:00 |
|
|
shuwenn
|
1d90b194b2
|
[model-gateway] bugfix: backward compatibility for GET endpoints (#15413)
|
2025-12-20 10:37:53 -10:00 |
|
|
Praneth Paruchuri
|
537ef18d17
|
[model-gateway] Optimize WASM Runtime with Instance Pooling and Component Caching (#15515)
|
2025-12-20 10:27:02 -10:00 |
|
|
Simo Lin
|
5529ab5895
|
[model-gateway] Implement RAII load guard with response body attachment (#15507)
|
2025-12-19 19:14:52 -08:00 |
|
|
kun-llfl
|
3c116d5e5a
|
[router] bugfix: cache_aware in grpc inbalance forward (#15473)
Signed-off-by: Kun(llfl) <i@imux.top>
|
2025-12-19 16:25:46 -08:00 |
|
|
Simo Lin
|
71a602883a
|
[model-gateway] simplify workflow engine backoff and reduce duplicate reads (#15505)
|
2025-12-19 16:23:02 -08:00 |
|
|
Simo Lin
|
61405b3d00
|
[model-gateway] Run workflow event subscribers concurrently (#15504)
|
2025-12-19 15:49:51 -08:00 |
|
|
Simo Lin
|
0adfc42b99
|
[model-gateway] Optimize workflow engine with pre-computed dependency graph (#15503)
|
2025-12-19 15:44:57 -08:00 |
|
|
Simo Lin
|
ba72e759ca
|
[model-gateway] Improve logging across core modules (#15497)
|
2025-12-19 13:35:28 -08:00 |
|
|
Simo Lin
|
bd16244d93
|
[model-gateway] Improve logging in policies module (#15496)
|
2025-12-19 13:02:57 -08:00 |
|
|
Simo Lin
|
b5eb0214ca
|
[model-gateway] Improve logging in data_connector module (#15495)
|
2025-12-19 12:57:18 -08:00 |
|
|
Simo Lin
|
d72e908ba5
|
[model-gateway] refactor: extract common graceful shutdown code before TLS branch (#15494)
|
2025-12-19 12:56:33 -08:00 |
|
|
Simo Lin
|
50cad014dc
|
[model-gateway] fix graceful shutdown for TLS/Non-TLS server (#15491)
|
2025-12-19 12:35:50 -08:00 |
|
|
Simo Lin
|
70607e55e8
|
[model-gateway] Replace PolicyRegistry RwLock with DashMap for lock-free policy lookups (#15361)
|
2025-12-18 00:11:20 -08:00 |
|
|
Simo Lin
|
53e151945a
|
[model-gateway] optimize worker registry and reduce lock contention in grpc client fetch (#15336)
|
2025-12-17 08:49:10 -10:00 |
|
|
Simo Lin
|
d747147a26
|
[model-gateway] reduce cpu overhead (#15316)
|
2025-12-17 09:42:01 -08:00 |
|
|
fzyzcjy
|
c4aed389da
|
Super tiny rename failure_count for consistency (#15186)
|
2025-12-17 21:21:07 +08:00 |
|
|
Simo Lin
|
3518b33178
|
[model-gateway] Remove legacy RouterMetrics and Rename SmgMetrics to Metrics and smg_labels to metrics_labels (#15160)
|
2025-12-15 04:57:42 -08:00 |
|