Simo Lin
|
b4ce7a6d71
|
[model-gateway] cache_aware eliminate String allocations in hot path (#16209)
|
2025-12-31 00:26:10 -08:00 |
|
Simo Lin
|
9940c6f592
|
[model-gateway]: ASCII byte comparison and probabilistic timestamp updates (#16181)
|
2025-12-30 12:28:01 -08:00 |
|
Simo Lin
|
4bc2f2e0bc
|
[model-gateway][docs] Add Classification API documentation (#16182)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-30 10:53:31 -08:00 |
|
Simo Lin
|
8c6f865afb
|
[model-gateway]: optimize prefix_match with zero-copy tenant and deferred char count (#16099)
|
2025-12-29 11:32:57 -08:00 |
|
Simo Lin
|
f39382c6ca
|
[model-gateway] Fix duplicate classify prefix in response ID (#16101)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 11:07:21 -08:00 |
|
Simo Lin
|
2ff289e2dc
|
[model-gateway] Generate UUID-based request IDs for embedding/classify (#16100)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 10:49:58 -08:00 |
|
Simo Lin
|
b2a3f0551a
|
[model-gateway] Wire classify pipeline to gRPC router (#16098)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 10:44:45 -08:00 |
|
Simo Lin
|
684e148e38
|
[model-gateway] Optimize INSERT with leaf-only timestamp updates (#16097)
|
2025-12-29 10:43:20 -08:00 |
|
Simo Lin
|
f44c4b3715
|
[model-gateway] Add classify pipeline stages and protocol types (#16094)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 10:23:36 -08:00 |
|
Simo Lin
|
ac78f96ec5
|
[model-gateway] Optimize radix tree timestamp updates for multi-tenant scaling (#16093)
|
2025-12-29 10:08:19 -08:00 |
|
Simo Lin
|
7f5879985c
|
[model-gateway] Improve tree benchmark with realistic multi-tenant scenarios (#14838)
|
2025-12-29 08:37:24 -08:00 |
|
Simo Lin
|
162d1cf9be
|
[model-gateway] Add classification model support infrastructure (#16061)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 08:34:05 -08:00 |
|
Simo Lin
|
c31f62722c
|
[model-gateway] fix tokenizer to match transformers special token handling (#16087)
|
2025-12-29 08:13:03 -08:00 |
|
Simo Lin
|
097330909f
|
[model-gateway] update WorkerRegistryStats with connection mode and circuit breaker info (#16046)
|
2025-12-28 23:14:57 -08:00 |
|
Simo Lin
|
ec8c831d5a
|
[model-gateway]: optimize metrics for minimal CPU and memory overhead (#16041)
|
2025-12-28 22:28:08 -08:00 |
|
Simo Lin
|
f13949e5a9
|
[model-gateway] perf: optimize observability logging for minimal CPU/memory overhead (#16039)
|
2025-12-28 21:20:46 -08:00 |
|
Simo Lin
|
c236a3fdf0
|
[model-gateway][CI] Display benchmark results in GitHub Actions summary (#16037)
|
2025-12-28 20:57:27 -08:00 |
|
Simo Lin
|
41a1d16b02
|
[model-gateway] Organize CLI arguments into logical groups for better --help output (#16035)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-12-28 20:25:53 -08:00 |
|
Simo Lin
|
9884c9fd4c
|
[model-gateway] Organize Rust CLI arguments into logical groups for better --help output (#16036)
|
2025-12-28 20:25:38 -08:00 |
|
fzyzcjy
|
8fab48952e
|
Tiny fix WASM test errors on machines with many cores (#15992)
|
2025-12-28 17:12:04 +08:00 |
|
fzyzcjy
|
7ccaec6467
|
Add micro benchmarks for manual policy (#15991)
|
2025-12-28 17:11:56 +08:00 |
|
fzyzcjy
|
26c5091217
|
Tiny extract PeriodicTask in router (#15988)
|
2025-12-28 14:07:13 +08:00 |
|
fzyzcjy
|
325a4c1945
|
Tiny add smg_manual_policy_cache_entries metric (#15987)
|
2025-12-28 14:07:07 +08:00 |
|
Ratish P
|
2af955e16c
|
[model-gateway]: remove unnecessary comment (#15947)
|
2025-12-27 13:33:02 -05:00 |
|
Simo Lin
|
3645ed0f73
|
[model-gateway] Add PrefixHash load balancing policy for KV cache-aware routing (#15935)
|
2025-12-27 05:58:05 -05:00 |
|
Ratish P
|
ca740a41f3
|
[model-gateway]: fix grpc embedding test (#15934)
|
2025-12-27 05:56:54 -05:00 |
|
Simo Lin
|
0e25aa439b
|
[model-gateway] optimize radix tree memory and reduce allocations (#15933)
|
2025-12-27 02:47:30 -08:00 |
|
Simo Lin
|
171912a9e3
|
[model-gateway] Add consistent hashing for ManualPolicy routing (#15907)
|
2025-12-26 22:53:03 -08:00 |
|
Simo Lin
|
4edee6954a
|
[model-gateway] add JWT/OIDC authentication for control plane APIs (#15850)
|
2025-12-26 18:30:00 -08:00 |
|
Simo Lin
|
acddb8e0ec
|
Revert embedding integration tests from 5f3a47d8a (#15914)
|
2025-12-26 20:29:03 -05:00 |
|
Simo Lin
|
caa95c7eb4
|
[model-gateway] update ManualPolicy with header-based routing (#15847)
|
2025-12-25 13:04:43 -08:00 |
|
fzyzcjy
|
6ef543f99d
|
Use X-SMG-Routing-Key header instead of json body and add tests (#15826)
|
2025-12-25 09:58:14 -08:00 |
|
fzyzcjy
|
ccf2330b28
|
Tiny fix missing record_router_upstream_response (#15811)
|
2025-12-25 15:20:43 +08:00 |
|
fzyzcjy
|
45adad37d0
|
Add manual routing policy for router (#15586)
|
2025-12-25 11:27:36 +08:00 |
|
fzyzcjy
|
1ba897f330
|
Tiny refactor select_workers API for future passing more information (#15596)
|
2025-12-25 10:31:10 +08:00 |
|
Simo Lin
|
5454d2a7ae
|
[model-gateway] release smg 0.3.0 (#15781)
|
2025-12-24 13:23:57 -08:00 |
|
Simo Lin
|
643aeefe6d
|
[model-gateway] Fix logging module name, parse endpoint context, and tokenizer factory (#15782)
|
2025-12-24 10:54:48 -08:00 |
|
Praneth Paruchuri
|
370bd27f3a
|
[model-gateway] Implement Zero-Copy Vision Tensor Access (#15750)
|
2025-12-24 09:15:44 -08:00 |
|
Simo Lin
|
2f7c629235
|
[model-gateway] Fix IGW routing and optimize RouterManager (#15741)
|
2025-12-24 11:28:53 -05:00 |
|
fzyzcjy
|
d6108166bc
|
Fix smg_http_requests_total semantics (#15655)
|
2025-12-24 17:41:01 +08:00 |
|
Arthur Cheng
|
f65fa04748
|
[model-gateway]Enable IGW mode with gRPC router and auto enable IGW when service discovery is turned on (#15459)
|
2025-12-24 00:27:58 -08:00 |
|
Simo Lin
|
9665574937
|
[docs] major SGL Model Gateway documentation update (#15715)
|
2025-12-23 20:26:09 -08:00 |
|
fzyzcjy
|
99d3bcdfed
|
[model-gateway] add back router worker health metric and fix init state (#15622)
|
2025-12-23 18:50:38 -08:00 |
|
fzyzcjy
|
4d64f15086
|
[mode;-gateway] add back fixes of incorrect metrics after worker removal (#15624)
|
2025-12-23 18:50:03 -08:00 |
|
Simo Lin
|
846953d9f1
|
[model-gateway] Add tokenize/detokenize HTTP endpoints and tokenizer management (#15702)
|
2025-12-23 17:32:07 -08:00 |
|
Simo Lin
|
aa6ac96635
|
[model-gateway] Fix tokenizer caching and improve error handling (#15695)
|
2025-12-23 10:33:38 -08:00 |
|
ratish
|
5f3a47d8a7
|
[model-gateway]: add gRPC router embeddings endpoint implementation (#15273)
|
2025-12-23 09:12:22 -08:00 |
|
Praneth Paruchuri
|
80ae2229d3
|
[model-gateway] Optimize router selection with lock-free snapshots (#15672)
|
2025-12-23 09:09:30 -08:00 |
|
Arthur Cheng
|
dd620987d1
|
[model-gateway] Replace tokenizer with tokenizer registry for dynamic tokenizer loading in gRPC router (#12968)
|
2025-12-23 07:58:47 -08:00 |
|
fzyzcjy
|
b736a1525a
|
Tiny add back missing router per attempt response metric (#15621)
|
2025-12-22 09:59:10 -08:00 |
|