 Simo Lin
|
0236522214
|
refactor: remove crate re-export aliases from lib.rs (#18737)
|
2026-02-12 09:17:24 -08:00 |
|
|
fzyzcjy
|
9c2530642c
|
Change routing policy API to be async to support more policies (#17048)
|
2026-01-17 17:12:51 +08:00 |
|
|
fzyzcjy
|
c824ddd581
|
Fix imbalancedness for manual policy min group mode for requests without routing id (#17044)
|
2026-01-17 11:58:35 +08:00 |
|
|
Chang Su
|
669d309a8b
|
[model-gateway] Consolidate "unknown" model id usage (#17186)
|
2026-01-15 21:02:40 -08:00 |
|
|
Tony LuandKun
|
5938c3b06a
|
[model-gateway] HA - Lightweight State Layer + gRPC Mesh (#14108)
Signed-off-by: Kun(llfl) <i@imux.top>
Signed-off-by: Tony Lu <tonylu@linux.alibaba.com>
Co-authored-by: Kun(llfl) <i@imux.top>
|
2026-01-13 17:03:39 -08:00 |
|
|
fzyzcjy
|
ff3ddb9d9b
|
Support min num routing keys in key-based load balancing policy (#16564)
|
2026-01-12 21:38:03 -08:00 |
|
|
fzyzcjy
|
9d3018f484
|
Support min load besides random routing key assignment policy in ManualPolicy (#16767)
|
2026-01-12 18:31:18 -08:00 |
|
|
fzyzcjy
|
3be1e734ee
|
[model-gateway] extract header extraction in policy and add (#16566)
|
2026-01-06 20:18:44 -08:00 |
|
|
fzyzcjy
|
00562ee14a
|
Support cache eviction for Manual Policy (#16263)
|
2026-01-02 08:52:01 +08:00 |
|
|
Simo Lin
|
851845579c
|
perf(tree): optimize input_char_count calculation (#16238)
|
2025-12-31 15:29:37 -08:00 |
|
|
Simo Lin
|
60d7279c46
|
test(tree): add comprehensive unit tests and fix input_char_count bug (#16228)
|
2025-12-31 07:31:39 -08:00 |
|
|
Simo Lin
|
b4ce7a6d71
|
[model-gateway] cache_aware eliminate String allocations in hot path (#16209)
|
2025-12-31 00:26:10 -08:00 |
|
|
Simo Lin
|
9940c6f592
|
[model-gateway]: ASCII byte comparison and probabilistic timestamp updates (#16181)
|
2025-12-30 12:28:01 -08:00 |
|
|
Simo Lin
|
8c6f865afb
|
[model-gateway]: optimize prefix_match with zero-copy tenant and deferred char count (#16099)
|
2025-12-29 11:32:57 -08:00 |
|
|
Simo LinandChang Su
|
f39382c6ca
|
[model-gateway] Fix duplicate classify prefix in response ID (#16101)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 11:07:21 -08:00 |
|
|
Simo Lin
|
684e148e38
|
[model-gateway] Optimize INSERT with leaf-only timestamp updates (#16097)
|
2025-12-29 10:43:20 -08:00 |
|
|
Simo Lin
|
ac78f96ec5
|
[model-gateway] Optimize radix tree timestamp updates for multi-tenant scaling (#16093)
|
2025-12-29 10:08:19 -08:00 |
|
|
fzyzcjy
|
26c5091217
|
Tiny extract PeriodicTask in router (#15988)
|
2025-12-28 14:07:13 +08:00 |
|
|
fzyzcjy
|
325a4c1945
|
Tiny add smg_manual_policy_cache_entries metric (#15987)
|
2025-12-28 14:07:07 +08:00 |
|
|
Simo Lin
|
3645ed0f73
|
[model-gateway] Add PrefixHash load balancing policy for KV cache-aware routing (#15935)
|
2025-12-27 05:58:05 -05:00 |
|
|
Simo Lin
|
0e25aa439b
|
[model-gateway] optimize radix tree memory and reduce allocations (#15933)
|
2025-12-27 02:47:30 -08:00 |
|
|
Simo Lin
|
171912a9e3
|
[model-gateway] Add consistent hashing for ManualPolicy routing (#15907)
|
2025-12-26 22:53:03 -08:00 |
|
|
Simo Lin
|
caa95c7eb4
|
[model-gateway] update ManualPolicy with header-based routing (#15847)
|
2025-12-25 13:04:43 -08:00 |
|
|
fzyzcjy
|
45adad37d0
|
Add manual routing policy for router (#15586)
|
2025-12-25 11:27:36 +08:00 |
|
|
fzyzcjy
|
1ba897f330
|
Tiny refactor select_workers API for future passing more information (#15596)
|
2025-12-25 10:31:10 +08:00 |
|
|
Simo Lin
|
aa6ac96635
|
[model-gateway] Fix tokenizer caching and improve error handling (#15695)
|
2025-12-23 10:33:38 -08:00 |
|
|
Arthur Cheng
|
dd620987d1
|
[model-gateway] Replace tokenizer with tokenizer registry for dynamic tokenizer loading in gRPC router (#12968)
|
2025-12-23 07:58:47 -08:00 |
|
|
Simo Lin
|
bd16244d93
|
[model-gateway] Improve logging in policies module (#15496)
|
2025-12-19 13:02:57 -08:00 |
|
|
Simo Lin
|
70607e55e8
|
[model-gateway] Replace PolicyRegistry RwLock with DashMap for lock-free policy lookups (#15361)
|
2025-12-18 00:11:20 -08:00 |
|
|
Simo Lin
|
3518b33178
|
[model-gateway] Remove legacy RouterMetrics and Rename SmgMetrics to Metrics and smg_labels to metrics_labels (#15160)
|
2025-12-15 04:57:42 -08:00 |
|
|
Simo Lin
|
7d8e42c979
|
[model-gateway] Fix metric emission gaps and name mismatch (#15093)
|
2025-12-13 20:16:30 -08:00 |
|
|
Praneth Paruchuri
|
e99ee0c695
|
[model-gateway] Fix incompatible metric comparison in PowerOfTwo policy (#14823)
|
2025-12-10 11:15:38 -08:00 |
|
|
Simo Lin
|
9ad02b799d
|
[Perf] Optimize radix tree for cache-aware load balancin (#14758)
|
2025-12-09 15:35:29 -08:00 |
|
|
fzyzcjy
|
817daba062
|
Tiny extract select_worker_min_load (#14648)
|
2025-12-08 21:22:10 -08:00 |
|
|
Simo Lin
|
edde5e5d40
|
[model-gateway] add OTEL integration to grpc router (#14671)
|
2025-12-08 14:45:05 -08:00 |
|
|
fzyzcjy
|
2e3946d889
|
Fix cache-aware router should pick min load instead of min tenant size (#14650)
|
2025-12-08 13:33:34 -08:00 |
|
|
Simo Lin
|
8550822d6b
|
[model-gateway] Optimize memory usage in HTTP router (#14667)
|
2025-12-08 13:10:31 -08:00 |
|
|
Simo Lin
|
d69ecc19b8
|
[model-gateway] reducing cpu overhead in various of places (#14658)
|
2025-12-08 09:44:40 -08:00 |
|
|
fzyzcjy
|
c08b780fe0
|
Super tiny remove unused select_worker_pair (#14609)
|
2025-12-07 20:32:38 -08:00 |
|
|
fzyzcjy
|
85d0ccfac0
|
Tiny fix missing policy decision recording (#14605)
|
2025-12-07 19:24:27 -08:00 |
|
|
Simo Lin
|
aff1238ef2
|
[model-gateway] reorganize metrics, logging, and otel to its own module (#14590)
|
2025-12-07 16:50:39 -08:00 |
|
|
Simo Lin
|
49dfa1d891
|
[model-gateway] change sgl-router to sgl-model-gateway (#14312)
|
2025-12-05 12:04:48 -08:00 |
|