Simo Lin
|
ac16d4450e
|
[smg][mesh] extract mesh to mesh crate to reduce compile time (#17907)
|
2026-01-28 17:58:42 -08:00 |
|
Praneth Paruchuri
|
6f009961bb
|
[model-gateway] Optimize HashRing construction to reduce heap allocations (#17575)
|
2026-01-27 22:46:40 -08:00 |
|
Praneth Paruchuri
|
897c35b457
|
[model-gateway] Optimize consistent hashing hot path to eliminate allocations (#17467)
|
2026-01-27 22:46:11 -08:00 |
|
shuwenn
|
a723d1c5ef
|
[model-gateway] ignore error for embeddings/classify in PD router (#15931)
|
2026-01-26 22:49:14 -05:00 |
|
Simo Lin
|
6756cf18c6
|
remove multimodal as this is completely dead code (#17750)
|
2026-01-26 00:07:40 -08:00 |
|
Simo Lin
|
d4adff31aa
|
update wasm endpoint (#17748)
|
2026-01-25 23:43:29 -08:00 |
|
Simo Lin
|
ed75136e85
|
remove self managed wasm as it has been replaced with official smg wa… (#17746)
|
2026-01-25 22:56:33 -08:00 |
|
Praneth Paruchuri
|
511961870f
|
[model-gateway] Optimize WASM cache lookups using SHA-256 (#17344)
|
2026-01-25 22:44:55 -08:00 |
|
Simo Lin
|
7890a96f97
|
remove self managed mcp as it has been replaced with official rmcp crate (#17740)
|
2026-01-25 22:17:06 -08:00 |
|
Simo Lin
|
46bc53a81b
|
[smg] import db crate to replace self managed one (#17727)
|
2026-01-25 16:54:52 -08:00 |
|
Simo Lin
|
52d0ca944b
|
[smg] import official wfaas crate to replace self managed one (#17724)
|
2026-01-25 15:20:58 -08:00 |
|
Simo Lin
|
bf139af37a
|
[smg] use already published auth crate for better compilation speed (#17723)
|
2026-01-25 14:52:42 -08:00 |
|
Simo Lin
|
97a36a72b7
|
[smg] remove dead tokenizer code (#17722)
|
2026-01-25 14:04:48 -08:00 |
|
Praneth Paruchuri
|
fc7096f80b
|
[model-gateway] Optimize special token search using Aho-Corasick (#17387)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-01-25 13:54:15 -08:00 |
|
Simo Lin
|
f5ac1ca10b
|
[smg] use official tokenizer crate instead of manually built one (#17721)
|
2026-01-25 13:35:16 -08:00 |
|
Simo Lin
|
2c1e2674cc
|
[misc] replace existing tool call code with new crate package (#17720)
|
2026-01-25 13:07:39 -08:00 |
|
Simo Lin
|
86c7bc6431
|
[smg] update crate for tools (#17717)
|
2026-01-25 12:56:42 -08:00 |
|
Simo Lin
|
7a1d7ab47b
|
remove self managed protocols as it has been replaced with official oai spec (#17711)
|
2026-01-25 06:38:13 -08:00 |
|
Simo Lin
|
8db2802b2d
|
update to use official openai protocol crate (#17710)
|
2026-01-25 06:35:59 -08:00 |
|
Simo Lin
|
6af22f8dbf
|
use published reasoning parser crate (#17709)
|
2026-01-25 06:15:03 -08:00 |
|
Jacob Gordon
|
42523d0364
|
ci(pre-commit): avoids extraneous codespell exclusions (#17590)
|
2026-01-22 09:55:23 -05:00 |
|
fzyzcjy
|
9c2530642c
|
Change routing policy API to be async to support more policies (#17048)
|
2026-01-17 17:12:51 +08:00 |
|
fzyzcjy
|
305c1a575a
|
Tiny remove unused code (#17047)
|
2026-01-17 11:58:43 +08:00 |
|
fzyzcjy
|
c824ddd581
|
Fix imbalancedness for manual policy min group mode for requests without routing id (#17044)
|
2026-01-17 11:58:35 +08:00 |
|
Chang Su
|
4229de3b13
|
[model-gateway] Refine TokenizerRegisty.load() to handle duplication (#17230)
|
2026-01-16 13:53:21 -08:00 |
|
Chang Su
|
a7f5677abe
|
refactor: unify registration through tokenizer_registration workflow (#17187)
|
2026-01-16 10:53:36 -08:00 |
|
Chang Su
|
669d309a8b
|
[model-gateway] Consolidate "unknown" model id usage (#17186)
|
2026-01-15 21:02:40 -08:00 |
|
jeff.ye
|
c020d30045
|
[Model-Gateway: grpc]: create tokenizer with chat template (#17052)
Signed-off-by: jeff.ye <jeff.ye@novita.ai>
|
2026-01-14 17:56:08 -08:00 |
|
Simo Lin
|
f091858304
|
[gRPC] Add GetLoads RPC for comprehensive load metrics (#17087)
|
2026-01-14 12:05:10 -08:00 |
|
Ziwen Zhao
|
cf25852a1d
|
[model-gateway] add --disable-health-check option to skip worker health probes (#17002)
|
2026-01-13 17:28:32 -08:00 |
|
Tony Lu
|
5938c3b06a
|
[model-gateway] HA - Lightweight State Layer + gRPC Mesh (#14108)
Signed-off-by: Kun(llfl) <i@imux.top>
Signed-off-by: Tony Lu <tonylu@linux.alibaba.com>
Co-authored-by: Kun(llfl) <i@imux.top>
|
2026-01-13 17:03:39 -08:00 |
|
Praneth Paruchuri
|
250477d2ac
|
[model-gateway] Optimize L1 cache insertion with incremental hashing and tokenization (#16259)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
|
2026-01-13 06:24:25 -08:00 |
|
fzyzcjy
|
ff3ddb9d9b
|
Support min num routing keys in key-based load balancing policy (#16564)
|
2026-01-12 21:38:03 -08:00 |
|
fzyzcjy
|
9d3018f484
|
Support min load besides random routing key assignment policy in ManualPolicy (#16767)
|
2026-01-12 18:31:18 -08:00 |
|
fzyzcjy
|
1f0e3d7fd8
|
Support tracking worker routing key loads in gateway (#16765)
|
2026-01-13 08:07:17 +08:00 |
|
Simo Lin
|
c6a64e9f69
|
[smg] fix type complexity for workflow run_if (#16981)
|
2026-01-12 15:41:52 -08:00 |
|
Simo Lin
|
e0ac559ae1
|
feat(workflow): add scheduled/delayed steps and conditional branching (#16980)
|
2026-01-12 15:06:12 -08:00 |
|
Simo Lin
|
6620548fd8
|
[model-gateway] make StateStore trait async for external persistence (#16979)
|
2026-01-12 12:55:59 -08:00 |
|
Simo Lin
|
6e158e55b4
|
[model-gateway] improve workflow engine code quality (#16977)
|
2026-01-12 12:14:46 -08:00 |
|
Simo Lin
|
ed729d22b3
|
[model-gateway] refactor workflow engine from type erasure to typed engines (#16973)
|
2026-01-12 10:47:00 -08:00 |
|
Simo Lin
|
fa51b85466
|
[model-gateway] convert workflow system to type-safe workflow data (#16970)
|
2026-01-12 10:06:36 -08:00 |
|
Simo Lin
|
d0092decb1
|
[model-gateway] Fix workflow engine race conditions and add graceful shutdown (#16963)
|
2026-01-12 06:37:39 -08:00 |
|
Wenyi Xu
|
c54c70ab62
|
[model-gateway] Improve Health Check Logging (#16930)
|
2026-01-11 23:01:29 -08:00 |
|
fzyzcjy
|
38b30c7b56
|
Tiny refactor age computation in router (#16850)
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
|
2026-01-11 21:42:43 -08:00 |
|
fzyzcjy
|
a1c48943d7
|
Tiny fix NoAvailableWorkers being a RetryError (#16896)
|
2026-01-11 18:53:17 -08:00 |
|
ybyang
|
503c3d9566
|
[model-gateway]: add qwen coder tool parser support xml format for qwen3 coder and microthinker (#12909)
|
2026-01-11 16:09:41 -08:00 |
|
Wenyi Xu
|
3c16c58619
|
[model-gateway] Add Redis support as a history backend (#16300)
|
2026-01-11 01:03:00 -08:00 |
|
Simo Lin
|
7c25687c9b
|
fix(gateway): rewrite gauge_histogram.rs for zero-allocation hot path (#16878)
|
2026-01-10 18:15:21 -08:00 |
|
fzyzcjy
|
1f9d4795a9
|
Tiny add gauge histogram abstraction for engine and router (#16848)
|
2026-01-10 11:45:25 +08:00 |
|
Praneth Paruchuri
|
bd1afeb568
|
[model-gateway] Restore response streaming by optimizing WASM middleware buffering (#16804)
|
2026-01-09 09:29:22 -08:00 |
|