Commit Graph
15 Commits
Author SHA1 Message Date
Simo Lin bd16244d93 [model-gateway] Improve logging in policies module (#15496) 2025-12-19 13:02:57 -08:00
Simo Lin 70607e55e8 [model-gateway] Replace PolicyRegistry RwLock with DashMap for lock-free policy lookups (#15361) 2025-12-18 00:11:20 -08:00
Simo Lin 3518b33178 [model-gateway] Remove legacy RouterMetrics and Rename SmgMetrics to Metrics and smg_labels to metrics_labels (#15160) 2025-12-15 04:57:42 -08:00
Simo Lin 7d8e42c979 [model-gateway] Fix metric emission gaps and name mismatch (#15093) 2025-12-13 20:16:30 -08:00
Praneth Paruchuri e99ee0c695 [model-gateway] Fix incompatible metric comparison in PowerOfTwo policy (#14823) 2025-12-10 11:15:38 -08:00
Simo Lin 9ad02b799d [Perf] Optimize radix tree for cache-aware load balancin (#14758) 2025-12-09 15:35:29 -08:00
fzyzcjy 817daba062 Tiny extract select_worker_min_load (#14648) 2025-12-08 21:22:10 -08:00
Simo Lin edde5e5d40 [model-gateway] add OTEL integration to grpc router (#14671) 2025-12-08 14:45:05 -08:00
fzyzcjy 2e3946d889 Fix cache-aware router should pick min load instead of min tenant size (#14650) 2025-12-08 13:33:34 -08:00
Simo Lin 8550822d6b [model-gateway] Optimize memory usage in HTTP router (#14667) 2025-12-08 13:10:31 -08:00
Simo Lin d69ecc19b8 [model-gateway] reducing cpu overhead in various of places (#14658) 2025-12-08 09:44:40 -08:00
fzyzcjy c08b780fe0 Super tiny remove unused select_worker_pair (#14609) 2025-12-07 20:32:38 -08:00
fzyzcjy 85d0ccfac0 Tiny fix missing policy decision recording (#14605) 2025-12-07 19:24:27 -08:00
Simo Lin aff1238ef2 [model-gateway] reorganize metrics, logging, and otel to its own module (#14590) 2025-12-07 16:50:39 -08:00
Simo Lin 49dfa1d891 [model-gateway] change sgl-router to sgl-model-gateway (#14312) 2025-12-05 12:04:48 -08:00