 shuwenn
|
a723d1c5ef
|
[model-gateway] ignore error for embeddings/classify in PD router (#15931)
|
2026-01-26 22:49:14 -05:00 |
|
|
fzyzcjy
|
9c2530642c
|
Change routing policy API to be async to support more policies (#17048)
|
2026-01-17 17:12:51 +08:00 |
|
|
Chang Su
|
669d309a8b
|
[model-gateway] Consolidate "unknown" model id usage (#17186)
|
2026-01-15 21:02:40 -08:00 |
|
|
fzyzcjy
|
ff3ddb9d9b
|
Support min num routing keys in key-based load balancing policy (#16564)
|
2026-01-12 21:38:03 -08:00 |
|
|
fzyzcjy
|
9d3018f484
|
Support min load besides random routing key assignment policy in ManualPolicy (#16767)
|
2026-01-12 18:31:18 -08:00 |
|
|
fzyzcjy
|
1f0e3d7fd8
|
Support tracking worker routing key loads in gateway (#16765)
|
2026-01-13 08:07:17 +08:00 |
|
|
fzyzcjy
|
9e3a032ad6
|
[smg] cleanup router RAII guards (#16560)
|
2026-01-08 16:39:53 -08:00 |
|
|
Praneth Paruchuri
|
b12258bfaa
|
[model-gateway] Optimize HTTP Router Fan-out: Replace Serial Execution with Concurrent Streams (#16042)
|
2026-01-04 23:38:45 -08:00 |
|
|
fzyzcjyandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
66dfb8c156
|
Tiny fix non-PD router http header missing whitelist (#16339)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-01-03 09:43:02 -08:00 |
|
|
Chang Su
|
c4edcac6d7
|
refactor(core): remove get_by_model_fast alias in worker_registry (#16313)
|
2026-01-02 16:04:23 -08:00 |
|
|
Chang Su
|
a2d4f58a96
|
fix(http): use 504 Gateway Timeout for upstream timeouts (#16320)
|
2026-01-02 16:01:44 -08:00 |
|
|
Chang Su
|
f66b091699
|
refactor(http): improve pd_types.rs documentation and style (#16319)
|
2026-01-02 16:01:11 -08:00 |
|
|
Simo Lin
|
f13949e5a9
|
[model-gateway] perf: optimize observability logging for minimal CPU/memory overhead (#16039)
|
2025-12-28 21:20:46 -08:00 |
|
|
Simo Lin
|
3645ed0f73
|
[model-gateway] Add PrefixHash load balancing policy for KV cache-aware routing (#15935)
|
2025-12-27 05:58:05 -05:00 |
|
|
Simo Lin
|
171912a9e3
|
[model-gateway] Add consistent hashing for ManualPolicy routing (#15907)
|
2025-12-26 22:53:03 -08:00 |
|
|
Simo Lin
|
caa95c7eb4
|
[model-gateway] update ManualPolicy with header-based routing (#15847)
|
2025-12-25 13:04:43 -08:00 |
|
|
fzyzcjy
|
6ef543f99d
|
Use X-SMG-Routing-Key header instead of json body and add tests (#15826)
|
2025-12-25 09:58:14 -08:00 |
|
|
fzyzcjy
|
ccf2330b28
|
Tiny fix missing record_router_upstream_response (#15811)
|
2025-12-25 15:20:43 +08:00 |
|
|
fzyzcjy
|
45adad37d0
|
Add manual routing policy for router (#15586)
|
2025-12-25 11:27:36 +08:00 |
|
|
fzyzcjy
|
1ba897f330
|
Tiny refactor select_workers API for future passing more information (#15596)
|
2025-12-25 10:31:10 +08:00 |
|
|
Simo Lin
|
643aeefe6d
|
[model-gateway] Fix logging module name, parse endpoint context, and tokenizer factory (#15782)
|
2025-12-24 10:54:48 -08:00 |
|
|
fzyzcjy
|
b736a1525a
|
Tiny add back missing router per attempt response metric (#15621)
|
2025-12-22 09:59:10 -08:00 |
|
|
Haoran Wang
|
ef001fb811
|
[model-gateway] /parse/easoning and parse/function_call for sgl-model-gateway (#15568)
|
2025-12-21 07:39:24 -10:00 |
|
|
Simo Lin
|
5529ab5895
|
[model-gateway] Implement RAII load guard with response body attachment (#15507)
|
2025-12-19 19:14:52 -08:00 |
|
|
Simo Lin
|
53e151945a
|
[model-gateway] optimize worker registry and reduce lock contention in grpc client fetch (#15336)
|
2025-12-17 08:49:10 -10:00 |
|
|
Simo Lin
|
d747147a26
|
[model-gateway] reduce cpu overhead (#15316)
|
2025-12-17 09:42:01 -08:00 |
|
|
Simo Lin
|
3518b33178
|
[model-gateway] Remove legacy RouterMetrics and Rename SmgMetrics to Metrics and smg_labels to metrics_labels (#15160)
|
2025-12-15 04:57:42 -08:00 |
|
|
Simo Lin
|
fb96669ff9
|
[model-gateway] Add Layer 3 worker metrics (smg_worker_*) (#15130)
|
2025-12-14 20:25:46 -08:00 |
|
|
fzyzcjy
|
62b3fdae43
|
Fix cache aware wrong routing caused by incorrect load tracking (#15101)
|
2025-12-15 10:03:54 +08:00 |
|
|
Simo Lin
|
b11af135eb
|
[model-gateway] feat(metrics): implement Layer 2 router metrics (smg_router_*) (#15124)
|
2025-12-14 10:34:12 -08:00 |
|
|
Simo Lin
|
74ea45cc4b
|
[model-gateway] optimize metric labels to avoid unnecessary allocations (#15095)
|
2025-12-13 21:10:45 -08:00 |
|
|
fzyzcjy
|
993278b488
|
Fix double decrease load (#15060)
|
2025-12-13 22:12:35 +08:00 |
|
|
fzyzcjy
|
9e9d910744
|
Fix load metric not updated when using guard (#15059)
|
2025-12-13 22:11:52 +08:00 |
|
|
fzyzcjy
|
bf17e769fe
|
Add sgl_router_attempt_http_responses_total for single attempt information (#15037)
|
2025-12-13 13:29:18 +08:00 |
|
|
fzyzcjy
|
31c23e5fe3
|
Provide more fine grained error reason for reqwest error (#15032)
|
2025-12-13 13:27:06 +08:00 |
|
|
fzyzcjy
|
06617a9ec8
|
Tiny change http router response format to unify (#15031)
|
2025-12-13 13:25:45 +08:00 |
|
|
fzyzcjy
|
ec242f516e
|
Super tiny extract route_typed_request_once (#14951)
|
2025-12-12 11:10:10 -08:00 |
|
|
fzyzcjy
|
e4873d0439
|
Super tiny remove sgl_router_active_workers (#14891)
|
2025-12-11 20:00:55 -08:00 |
|
|
Jimmy
|
2c4d376d23
|
[SMG][DS32][fix] support dsv32, add role developer (#14307)
|
2025-12-11 08:09:09 -08:00 |
|
|
Simo Lin
|
cd1c1fa5bc
|
[model-gateway] fix annotation error and code formating (#14910)
|
2025-12-11 07:06:32 -08:00 |
|
|
fzyzcjy
|
ba88f1cace
|
Tiny clean router load report logic (#14889)
|
2025-12-11 06:50:32 -08:00 |
|
|
Simo Lin
|
c032b559a8
|
[model-gateway] adds default implementations to RouterTrait in mod.rs (#14841)
|
2025-12-10 11:46:35 -08:00 |
|
|
fzyzcjy
|
3d82c0f17f
|
[model-gateway] support engine response http status statistics in router (#14712)
|
2025-12-10 08:30:53 -08:00 |
|
|
Simo Lin
|
8550822d6b
|
[model-gateway] Optimize memory usage in HTTP router (#14667)
|
2025-12-08 13:10:31 -08:00 |
|
|
Simo Lin
|
d69ecc19b8
|
[model-gateway] reducing cpu overhead in various of places (#14658)
|
2025-12-08 09:44:40 -08:00 |
|
|
Simo Lin
|
8fbf7dd56f
|
[model-gateway] refactor otel to be more efficient (#14604)
|
2025-12-07 20:09:25 -08:00 |
|
|
Simo Lin
|
aff1238ef2
|
[model-gateway] reorganize metrics, logging, and otel to its own module (#14590)
|
2025-12-07 16:50:39 -08:00 |
|
|
Feng Su
|
e12c6b320f
|
[model-gateway][tracing]: implement request tracing using OpenTelemetry with trace context propagation (HTTP) (#13897)
|
2025-12-06 05:59:04 -08:00 |
|
|
Simo Lin
|
49dfa1d891
|
[model-gateway] change sgl-router to sgl-model-gateway (#14312)
|
2025-12-05 12:04:48 -08:00 |
|