Tony Lu
|
5938c3b06a
|
[model-gateway] HA - Lightweight State Layer + gRPC Mesh (#14108)
Signed-off-by: Kun(llfl) <i@imux.top>
Signed-off-by: Tony Lu <tonylu@linux.alibaba.com>
Co-authored-by: Kun(llfl) <i@imux.top>
|
2026-01-13 17:03:39 -08:00 |
|
fzyzcjy
|
1f0e3d7fd8
|
Support tracking worker routing key loads in gateway (#16765)
|
2026-01-13 08:07:17 +08:00 |
|
fzyzcjy
|
38b30c7b56
|
Tiny refactor age computation in router (#16850)
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
|
2026-01-11 21:42:43 -08:00 |
|
Simo Lin
|
7c25687c9b
|
fix(gateway): rewrite gauge_histogram.rs for zero-allocation hot path (#16878)
|
2026-01-10 18:15:21 -08:00 |
|
fzyzcjy
|
1f9d4795a9
|
Tiny add gauge histogram abstraction for engine and router (#16848)
|
2026-01-10 11:45:25 +08:00 |
|
Chang Su
|
b98bd9a5fb
|
[model-gateway] Tighten visibility in modules and remove unused re-exports (#16524)
|
2026-01-05 17:38:50 -08:00 |
|
fzyzcjy
|
1e41069ad1
|
Fix age bucket rendering issue (#16492)
|
2026-01-05 09:29:14 -08:00 |
|
Simo Lin
|
4436dc0f6c
|
[model-gateway] improve lock contention and allocation in middleware (#16405)
|
2026-01-04 10:40:07 -08:00 |
|
fzyzcjy
|
c88aaf22c7
|
Support in-flight request age metrics for router (#16341)
|
2026-01-04 07:38:53 +08:00 |
|
Chang Su
|
d8ac5eecf7
|
[model-gateway] bug fix on module name (#16332)
|
2026-01-02 21:56:18 -08:00 |
|
Chang Su
|
b23e7ed13c
|
refactor(gateway): shorten logging targets from sgl_model_gateway to smg (#16328)
|
2026-01-02 21:12:33 -08:00 |
|
Simo Lin
|
f44c4b3715
|
[model-gateway] Add classify pipeline stages and protocol types (#16094)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 10:23:36 -08:00 |
|
Simo Lin
|
ec8c831d5a
|
[model-gateway]: optimize metrics for minimal CPU and memory overhead (#16041)
|
2025-12-28 22:28:08 -08:00 |
|
Simo Lin
|
f13949e5a9
|
[model-gateway] perf: optimize observability logging for minimal CPU/memory overhead (#16039)
|
2025-12-28 21:20:46 -08:00 |
|
fzyzcjy
|
325a4c1945
|
Tiny add smg_manual_policy_cache_entries metric (#15987)
|
2025-12-28 14:07:07 +08:00 |
|
Simo Lin
|
3645ed0f73
|
[model-gateway] Add PrefixHash load balancing policy for KV cache-aware routing (#15935)
|
2025-12-27 05:58:05 -05:00 |
|
Simo Lin
|
171912a9e3
|
[model-gateway] Add consistent hashing for ManualPolicy routing (#15907)
|
2025-12-26 22:53:03 -08:00 |
|
Simo Lin
|
caa95c7eb4
|
[model-gateway] update ManualPolicy with header-based routing (#15847)
|
2025-12-25 13:04:43 -08:00 |
|
fzyzcjy
|
45adad37d0
|
Add manual routing policy for router (#15586)
|
2025-12-25 11:27:36 +08:00 |
|
fzyzcjy
|
d6108166bc
|
Fix smg_http_requests_total semantics (#15655)
|
2025-12-24 17:41:01 +08:00 |
|
fzyzcjy
|
99d3bcdfed
|
[model-gateway] add back router worker health metric and fix init state (#15622)
|
2025-12-23 18:50:38 -08:00 |
|
fzyzcjy
|
4d64f15086
|
[mode;-gateway] add back fixes of incorrect metrics after worker removal (#15624)
|
2025-12-23 18:50:03 -08:00 |
|
Simo Lin
|
d747147a26
|
[model-gateway] reduce cpu overhead (#15316)
|
2025-12-17 09:42:01 -08:00 |
|
Simo Lin
|
3518b33178
|
[model-gateway] Remove legacy RouterMetrics and Rename SmgMetrics to Metrics and smg_labels to metrics_labels (#15160)
|
2025-12-15 04:57:42 -08:00 |
|
Simo Lin
|
7b9156c773
|
[model-gateway] add mcp and discovery metrics (#15156)
|
2025-12-14 22:54:53 -08:00 |
|
Simo Lin
|
bd9c3a47d6
|
[model-gateway] Add streaming metrics for harmony gRPC router (#15147)
|
2025-12-14 20:54:43 -08:00 |
|
Simo Lin
|
0612175ce1
|
[model-gateway] add streaming metrics (TTFT, TPOT, tokens, duration) for gRPC router (#15125)
|
2025-12-14 14:39:10 -08:00 |
|
Simo Lin
|
b11af135eb
|
[model-gateway] feat(metrics): implement Layer 2 router metrics (smg_router_*) (#15124)
|
2025-12-14 10:34:12 -08:00 |
|
Simo Lin
|
47633c191f
|
[model-gateway] Add new SMG metrics architecture with 6 layers (#15106)
|
2025-12-14 08:37:19 -08:00 |
|
fzyzcjy
|
54df514bac
|
Avoid confusing zero value metric when worker is removed (#15096)
|
2025-12-14 13:57:20 +08:00 |
|
Simo Lin
|
74ea45cc4b
|
[model-gateway] optimize metric labels to avoid unnecessary allocations (#15095)
|
2025-12-13 21:10:45 -08:00 |
|
Simo Lin
|
0fa044ad63
|
[model-gateway] Add circuit breaker and discovery watcher metrics (#15094)
|
2025-12-13 20:47:04 -08:00 |
|
Simo Lin
|
7d8e42c979
|
[model-gateway] Fix metric emission gaps and name mismatch (#15093)
|
2025-12-13 20:16:30 -08:00 |
|
Simo Lin
|
20ce9938b5
|
[model-gateway] Remove unused TokenizerMetrics to reduce CPU overhead (#15087)
|
2025-12-13 18:54:18 -08:00 |
|
fzyzcjy
|
bf17e769fe
|
Add sgl_router_attempt_http_responses_total for single attempt information (#15037)
|
2025-12-13 13:29:18 +08:00 |
|
fzyzcjy
|
9a5d6a84ab
|
Add error code in prometheus metrics and add X-SMG-Error-Code header (#15036)
|
2025-12-13 13:28:32 +08:00 |
|
fzyzcjy
|
a7fa31ff17
|
Super tiny move error.rs (#14944)
|
2025-12-12 14:25:30 +08:00 |
|
fzyzcjy
|
bd91f8824a
|
Super tiny remove non-updated sgl_router_worker_load (#14888)
|
2025-12-12 13:19:30 +08:00 |
|
fzyzcjy
|
b47adb807e
|
Tiny add e2e http request arrival metric (#14893)
|
2025-12-12 13:18:23 +08:00 |
|
fzyzcjy
|
b2b5bdb06e
|
Tiny add router e2e duration histogram (#14892)
|
2025-12-12 13:18:13 +08:00 |
|
fzyzcjy
|
e4873d0439
|
Super tiny remove sgl_router_active_workers (#14891)
|
2025-12-11 20:00:55 -08:00 |
|
fzyzcjy
|
3d82c0f17f
|
[model-gateway] support engine response http status statistics in router (#14712)
|
2025-12-10 08:30:53 -08:00 |
|
fzyzcjy
|
617e9b3bf8
|
[model-gateway] support customizing Prometheus duration buckets (#14716)
|
2025-12-10 08:30:03 -08:00 |
|
fzyzcjy
|
6634f67bd5
|
Fix router keep nonzero metrics after worker is deleted (#14819)
|
2025-12-10 22:29:30 +08:00 |
|
fzyzcjy
|
af20657cd4
|
Tiny support sgl-router http response status code metrics (#14689)
|
2025-12-08 19:54:35 -08:00 |
|
Simo Lin
|
edde5e5d40
|
[model-gateway] add OTEL integration to grpc router (#14671)
|
2025-12-08 14:45:05 -08:00 |
|
Simo Lin
|
8fbf7dd56f
|
[model-gateway] refactor otel to be more efficient (#14604)
|
2025-12-07 20:09:25 -08:00 |
|
Simo Lin
|
aff1238ef2
|
[model-gateway] reorganize metrics, logging, and otel to its own module (#14590)
|
2025-12-07 16:50:39 -08:00 |
|