 fzyzcjy
|
9e3a032ad6
|
[smg] cleanup router RAII guards (#16560)
|
2026-01-08 16:39:53 -08:00 |
|
|
Simo LinandChang Su
|
1bc7aa5801
|
[smg] update gRPC proto to match upstream changes (#16764)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2026-01-08 16:33:09 -08:00 |
|
|
Chang Su
|
05b54b6d7b
|
[router][grpc] Replace Vec<(String, String, String)> with ExtractedToolCall (#16598)
|
2026-01-06 16:32:59 -08:00 |
|
|
Chang Su
|
21da2dc1ce
|
[grpc] Unify ResponsesContext and HarmonyResponsesContext (#16549)
|
2026-01-06 07:28:06 -08:00 |
|
|
Chang Su
|
9bf76c11a9
|
[responses API] Add list_tools_for_servers and threading server_keys in routers (#16540)
|
2026-01-05 22:42:22 -08:00 |
|
|
Chang Su
|
4cf2bbd084
|
[router] Remove deadcode and add note for unused API completeness methods (#16528)
|
2026-01-05 19:53:28 -08:00 |
|
|
Chang Su
|
1751c75b5d
|
[model-gateway] Tighten visibility across data_connector and grpc module (#16516)
|
2026-01-05 12:31:32 -08:00 |
|
|
Chang Su
|
5a2b1ed407
|
[grpc] Refactor grpc/regular/responses (#16509)
|
2026-01-05 10:21:34 -08:00 |
|
|
Chang Su
|
454dc9e242
|
[model-gateway][grpc] Refactor harmony/responses.rs (#16508)
|
2026-01-05 10:21:05 -08:00 |
|
|
Simo Lin
|
cf6800f6cc
|
[model-gateway] optimize Vec and HashMap allocations in responses api (#16406)
|
2026-01-04 10:07:55 -08:00 |
|
|
Chang Su
|
8d58b3dcdd
|
[model-gateway] Clear architectual debt in responses API (#16359)
|
2026-01-03 21:35:28 -08:00 |
|
|
Simo Lin
|
65b0b5b248
|
fix(logging): use Display format for model_id instead of Debug (#16337)
|
2026-01-03 00:21:39 -08:00 |
|
|
Chang Su
|
d8ac5eecf7
|
[model-gateway] bug fix on module name (#16332)
|
2026-01-02 21:56:18 -08:00 |
|
|
Chang Su
|
b23e7ed13c
|
refactor(gateway): shorten logging targets from sgl_model_gateway to smg (#16328)
|
2026-01-02 21:12:33 -08:00 |
|
|
Chang Su
|
c4edcac6d7
|
refactor(core): remove get_by_model_fast alias in worker_registry (#16313)
|
2026-01-02 16:04:23 -08:00 |
|
|
Simo LinandChang Su
|
f39382c6ca
|
[model-gateway] Fix duplicate classify prefix in response ID (#16101)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 11:07:21 -08:00 |
|
|
Simo LinandChang Su
|
2ff289e2dc
|
[model-gateway] Generate UUID-based request IDs for embedding/classify (#16100)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 10:49:58 -08:00 |
|
|
Simo LinandChang Su
|
b2a3f0551a
|
[model-gateway] Wire classify pipeline to gRPC router (#16098)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 10:44:45 -08:00 |
|
|
Simo LinandChang Su
|
f44c4b3715
|
[model-gateway] Add classify pipeline stages and protocol types (#16094)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 10:23:36 -08:00 |
|
|
Simo LinandChang Su
|
162d1cf9be
|
[model-gateway] Add classification model support infrastructure (#16061)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 08:34:05 -08:00 |
|
|
Simo Lin
|
c31f62722c
|
[model-gateway] fix tokenizer to match transformers special token handling (#16087)
|
2025-12-29 08:13:03 -08:00 |
|
|
Simo Lin
|
3645ed0f73
|
[model-gateway] Add PrefixHash load balancing policy for KV cache-aware routing (#15935)
|
2025-12-27 05:58:05 -05:00 |
|
|
Simo Lin
|
171912a9e3
|
[model-gateway] Add consistent hashing for ManualPolicy routing (#15907)
|
2025-12-26 22:53:03 -08:00 |
|
|
Simo Lin
|
caa95c7eb4
|
[model-gateway] update ManualPolicy with header-based routing (#15847)
|
2025-12-25 13:04:43 -08:00 |
|
|
fzyzcjy
|
6ef543f99d
|
Use X-SMG-Routing-Key header instead of json body and add tests (#15826)
|
2025-12-25 09:58:14 -08:00 |
|
|
fzyzcjy
|
45adad37d0
|
Add manual routing policy for router (#15586)
|
2025-12-25 11:27:36 +08:00 |
|
|
fzyzcjy
|
1ba897f330
|
Tiny refactor select_workers API for future passing more information (#15596)
|
2025-12-25 10:31:10 +08:00 |
|
|
Simo Lin
|
846953d9f1
|
[model-gateway] Add tokenize/detokenize HTTP endpoints and tokenizer management (#15702)
|
2025-12-23 17:32:07 -08:00 |
|
|
Simo Lin
|
aa6ac96635
|
[model-gateway] Fix tokenizer caching and improve error handling (#15695)
|
2025-12-23 10:33:38 -08:00 |
|
|
ratish
|
5f3a47d8a7
|
[model-gateway]: add gRPC router embeddings endpoint implementation (#15273)
|
2025-12-23 09:12:22 -08:00 |
|
|
Arthur Cheng
|
dd620987d1
|
[model-gateway] Replace tokenizer with tokenizer registry for dynamic tokenizer loading in gRPC router (#12968)
|
2025-12-23 07:58:47 -08:00 |
|
|
fzyzcjy
|
6984837d82
|
Fix router gRPC mode launch error caused by async loading (#15368)
|
2025-12-22 09:39:01 -08:00 |
|
|
Simo Lin
|
122c250336
|
[model-gateway] add retry and circuit breaker support to gRPC routers (#15585)
|
2025-12-21 17:12:36 -08:00 |
|
|
Simo Lin
|
5529ab5895
|
[model-gateway] Implement RAII load guard with response body attachment (#15507)
|
2025-12-19 19:14:52 -08:00 |
|
|
kun-llfl
|
3c116d5e5a
|
[router] bugfix: cache_aware in grpc inbalance forward (#15473)
Signed-off-by: Kun(llfl) <i@imux.top>
|
2025-12-19 16:25:46 -08:00 |
|
|
Simo Lin
|
ba72e759ca
|
[model-gateway] Improve logging across core modules (#15497)
|
2025-12-19 13:35:28 -08:00 |
|
|
Simo Lin
|
d747147a26
|
[model-gateway] reduce cpu overhead (#15316)
|
2025-12-17 09:42:01 -08:00 |
|
|
Simo Lin
|
3518b33178
|
[model-gateway] Remove legacy RouterMetrics and Rename SmgMetrics to Metrics and smg_labels to metrics_labels (#15160)
|
2025-12-15 04:57:42 -08:00 |
|
|
Simo Lin
|
7b9156c773
|
[model-gateway] add mcp and discovery metrics (#15156)
|
2025-12-14 22:54:53 -08:00 |
|
|
Simo Lin
|
bd9c3a47d6
|
[model-gateway] Add streaming metrics for harmony gRPC router (#15147)
|
2025-12-14 20:54:43 -08:00 |
|
|
Simo Lin
|
fb96669ff9
|
[model-gateway] Add Layer 3 worker metrics (smg_worker_*) (#15130)
|
2025-12-14 20:25:46 -08:00 |
|
|
Simo Lin
|
0612175ce1
|
[model-gateway] add streaming metrics (TTFT, TPOT, tokens, duration) for gRPC router (#15125)
|
2025-12-14 14:39:10 -08:00 |
|
|
Simo Lin
|
b11af135eb
|
[model-gateway] feat(metrics): implement Layer 2 router metrics (smg_router_*) (#15124)
|
2025-12-14 10:34:12 -08:00 |
|
|
fzyzcjy
|
e79ca95961
|
Tiny unify grpc existing error responses into new format (#15030)
|
2025-12-13 13:25:10 +08:00 |
|
|
fzyzcjy
|
9d3b411c56
|
Add code field and unify error responses for router (#15028)
|
2025-12-13 13:21:34 +08:00 |
|
|
fzyzcjy
|
a7fa31ff17
|
Super tiny move error.rs (#14944)
|
2025-12-12 14:25:30 +08:00 |
|
|
Jimmy
|
2c4d376d23
|
[SMG][DS32][fix] support dsv32, add role developer (#14307)
|
2025-12-11 08:09:09 -08:00 |
|
|
Simo Lin
|
c032b559a8
|
[model-gateway] adds default implementations to RouterTrait in mod.rs (#14841)
|
2025-12-10 11:46:35 -08:00 |
|
|
Simo Lin
|
edde5e5d40
|
[model-gateway] add OTEL integration to grpc router (#14671)
|
2025-12-08 14:45:05 -08:00 |
|
|
Simo Lin
|
39f9a9c2a5
|
[model-gateway] reduce cpu overhead in grpc router (#14663)
|
2025-12-08 11:54:56 -08:00 |
|