Kangyan-Zhou
|
0eb64c1e72
|
[smg] Extract tokenizer_path from /model_info into discovered labels (#19905)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-03-04 17:12:20 -08:00 |
|
Eric Zhang
|
e6411ba315
|
Increase max_concurrent_jobs in job queue (#19797)
|
2026-03-03 15:54:51 -08:00 |
|
Simo Lin
|
0236522214
|
refactor: remove crate re-export aliases from lib.rs (#18737)
|
2026-02-12 09:17:24 -08:00 |
|
Praneth Paruchuri
|
6f009961bb
|
[model-gateway] Optimize HashRing construction to reduce heap allocations (#17575)
|
2026-01-27 22:46:40 -08:00 |
|
Simo Lin
|
8db2802b2d
|
update to use official openai protocol crate (#17710)
|
2026-01-25 06:35:59 -08:00 |
|
fzyzcjy
|
305c1a575a
|
Tiny remove unused code (#17047)
|
2026-01-17 11:58:43 +08:00 |
|
Chang Su
|
4229de3b13
|
[model-gateway] Refine TokenizerRegisty.load() to handle duplication (#17230)
|
2026-01-16 13:53:21 -08:00 |
|
Chang Su
|
a7f5677abe
|
refactor: unify registration through tokenizer_registration workflow (#17187)
|
2026-01-16 10:53:36 -08:00 |
|
jeff.ye
|
c020d30045
|
[Model-Gateway: grpc]: create tokenizer with chat template (#17052)
Signed-off-by: jeff.ye <jeff.ye@novita.ai>
|
2026-01-14 17:56:08 -08:00 |
|
Ziwen Zhao
|
cf25852a1d
|
[model-gateway] add --disable-health-check option to skip worker health probes (#17002)
|
2026-01-13 17:28:32 -08:00 |
|
Tony Lu
|
5938c3b06a
|
[model-gateway] HA - Lightweight State Layer + gRPC Mesh (#14108)
Signed-off-by: Kun(llfl) <i@imux.top>
Signed-off-by: Tony Lu <tonylu@linux.alibaba.com>
Co-authored-by: Kun(llfl) <i@imux.top>
|
2026-01-13 17:03:39 -08:00 |
|
fzyzcjy
|
1f0e3d7fd8
|
Support tracking worker routing key loads in gateway (#16765)
|
2026-01-13 08:07:17 +08:00 |
|
Simo Lin
|
ed729d22b3
|
[model-gateway] refactor workflow engine from type erasure to typed engines (#16973)
|
2026-01-12 10:47:00 -08:00 |
|
Simo Lin
|
fa51b85466
|
[model-gateway] convert workflow system to type-safe workflow data (#16970)
|
2026-01-12 10:06:36 -08:00 |
|
Wenyi Xu
|
c54c70ab62
|
[model-gateway] Improve Health Check Logging (#16930)
|
2026-01-11 23:01:29 -08:00 |
|
fzyzcjy
|
a1c48943d7
|
Tiny fix NoAvailableWorkers being a RetryError (#16896)
|
2026-01-11 18:53:17 -08:00 |
|
fzyzcjy
|
9e3a032ad6
|
[smg] cleanup router RAII guards (#16560)
|
2026-01-08 16:39:53 -08:00 |
|
Chang Su
|
b98bd9a5fb
|
[model-gateway] Tighten visibility in modules and remove unused re-exports (#16524)
|
2026-01-05 17:38:50 -08:00 |
|
Ratish P
|
f02d82211a
|
[model-gateway]: move PD configuration conflict checks to model gateway (#16088)
|
2026-01-05 14:16:07 +08:00 |
|
Simo Lin
|
4436dc0f6c
|
[model-gateway] improve lock contention and allocation in middleware (#16405)
|
2026-01-04 10:40:07 -08:00 |
|
Chang Su
|
d8ac5eecf7
|
[model-gateway] bug fix on module name (#16332)
|
2026-01-02 21:56:18 -08:00 |
|
Chang Su
|
24c91001cf
|
[model-gateway] code clean up in tokenizer register step workflow (#16316)
|
2026-01-02 16:05:10 -08:00 |
|
Chang Su
|
c4edcac6d7
|
refactor(core): remove get_by_model_fast alias in worker_registry (#16313)
|
2026-01-02 16:04:23 -08:00 |
|
Chang Su
|
e93433892b
|
refactor(steps): consolidate duplicate strip_protocol function (#16318)
|
2026-01-02 16:02:42 -08:00 |
|
Chang Su
|
2f623368ab
|
refactor(core): use idiomatic .min() in calculate_delay (#16314)
|
2026-01-02 15:46:13 -08:00 |
|
Chang Su
|
bd9a2ced47
|
refactor(core): use thiserror for WorkerError (#16315)
|
2026-01-02 15:33:07 -08:00 |
|
Chang Su
|
0ca417d9ae
|
chore(core): remove unused serde_json import in worker.rs (#16312)
|
2026-01-02 15:31:41 -08:00 |
|
Simo Lin
|
162d1cf9be
|
[model-gateway] Add classification model support infrastructure (#16061)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 08:34:05 -08:00 |
|
Simo Lin
|
097330909f
|
[model-gateway] update WorkerRegistryStats with connection mode and circuit breaker info (#16046)
|
2025-12-28 23:14:57 -08:00 |
|
Simo Lin
|
171912a9e3
|
[model-gateway] Add consistent hashing for ManualPolicy routing (#15907)
|
2025-12-26 22:53:03 -08:00 |
|
fzyzcjy
|
99d3bcdfed
|
[model-gateway] add back router worker health metric and fix init state (#15622)
|
2025-12-23 18:50:38 -08:00 |
|
fzyzcjy
|
4d64f15086
|
[mode;-gateway] add back fixes of incorrect metrics after worker removal (#15624)
|
2025-12-23 18:50:03 -08:00 |
|
Simo Lin
|
846953d9f1
|
[model-gateway] Add tokenize/detokenize HTTP endpoints and tokenizer management (#15702)
|
2025-12-23 17:32:07 -08:00 |
|
Simo Lin
|
aa6ac96635
|
[model-gateway] Fix tokenizer caching and improve error handling (#15695)
|
2025-12-23 10:33:38 -08:00 |
|
Arthur Cheng
|
dd620987d1
|
[model-gateway] Replace tokenizer with tokenizer registry for dynamic tokenizer loading in gRPC router (#12968)
|
2025-12-23 07:58:47 -08:00 |
|
Simo Lin
|
b9d786059f
|
[model-gateway] refactor WorkerManager with fan_out helper and thin handlers (#15583)
|
2025-12-21 15:29:37 -08:00 |
|
Simo Lin
|
1354063a48
|
[model-gateway] add WorkerService abstraction for worker business logic (#15580)
|
2025-12-21 14:35:35 -08:00 |
|
Simo Lin
|
e1dcd0df3e
|
[model-gateway] minor code clean up (#15578)
|
2025-12-21 11:00:32 -10:00 |
|
shuwenn
|
1d9ba2ce4d
|
[model-gateway] Use UUIDs for router-managed worker resources (#15540)
|
2025-12-21 07:40:03 -10:00 |
|
shuwenn
|
1d90b194b2
|
[model-gateway] bugfix: backward compatibility for GET endpoints (#15413)
|
2025-12-20 10:37:53 -10:00 |
|
Simo Lin
|
5529ab5895
|
[model-gateway] Implement RAII load guard with response body attachment (#15507)
|
2025-12-19 19:14:52 -08:00 |
|
Simo Lin
|
53e151945a
|
[model-gateway] optimize worker registry and reduce lock contention in grpc client fetch (#15336)
|
2025-12-17 08:49:10 -10:00 |
|
Simo Lin
|
d747147a26
|
[model-gateway] reduce cpu overhead (#15316)
|
2025-12-17 09:42:01 -08:00 |
|
fzyzcjy
|
c4aed389da
|
Super tiny rename failure_count for consistency (#15186)
|
2025-12-17 21:21:07 +08:00 |
|
Simo Lin
|
3518b33178
|
[model-gateway] Remove legacy RouterMetrics and Rename SmgMetrics to Metrics and smg_labels to metrics_labels (#15160)
|
2025-12-15 04:57:42 -08:00 |
|
fzyzcjy
|
89ad390843
|
Fix num running requests (load) wrong cleared for ongoing requests (#15116)
|
2025-12-15 17:40:04 +08:00 |
|
Simo Lin
|
7b9156c773
|
[model-gateway] add mcp and discovery metrics (#15156)
|
2025-12-14 22:54:53 -08:00 |
|
Simo Lin
|
fb96669ff9
|
[model-gateway] Add Layer 3 worker metrics (smg_worker_*) (#15130)
|
2025-12-14 20:25:46 -08:00 |
|
fzyzcjy
|
62b3fdae43
|
Fix cache aware wrong routing caused by incorrect load tracking (#15101)
|
2025-12-15 10:03:54 +08:00 |
|
fzyzcjy
|
4449c17011
|
[model-gateway] fix circuit breaker metrics (#15099)
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
|
2025-12-14 15:43:04 -08:00 |
|