 Simo Lin
|
e7d6027e4a
|
[model-gateway] add ModelCard support to WorkerMetadata (#14243)
|
2025-12-01 13:04:09 -08:00 |
|
|
Simo Lin
|
9325f94534
|
[model-gateway] add ModelCard and ProviderType for model configuration (#14237)
|
2025-12-01 09:47:51 -08:00 |
|
|
Simo Lin
|
92ca629582
|
[model-gateway] add ModelType bitflags and Endpoint enum for worker (#14230)
|
2025-12-01 08:14:44 -08:00 |
|
|
Simo Lin
|
79b389da6b
|
[model-gateway] refactor oai router 1/n (#14228)
|
2025-12-01 07:16:35 -08:00 |
|
|
Simo Lin
|
e9e90460ed
|
[model-gateway] allow refill rate to be zero (#14030)
|
2025-11-26 17:15:27 -08:00 |
|
|
Simo Lin
|
69a03bc3f7
|
[ci] allow manual label to trigger ci in rust, change ci order (#14016)
|
2025-11-26 11:38:50 -08:00 |
|
|
Simo Lin
|
7e964b5198
|
[ci] mark skip as success instead of failure (#14014)
|
2025-11-26 10:39:38 -08:00 |
|
|
Simo Lin
|
a0d9f6cdd8
|
[model-gateway] fix xpu ci (#14012)
|
2025-11-26 10:33:59 -08:00 |
|
|
Simo Lin
|
4852aa054c
|
[misc] add llama3.1 chat template (#13935)
|
2025-11-25 09:31:54 -08:00 |
|
|
Simo Lin
|
9b4b344115
|
[model-gateway] add grpc server code owner (#13865)
|
2025-11-24 12:29:57 -08:00 |
|
|
Simo Lin
|
9054e844ea
|
remove package json which is not used (#13810)
|
2025-11-23 15:00:30 -08:00 |
|
|
Simo Lin
|
5354d7b7fd
|
[model-gateway] clean up router manager function order (#13776)
|
2025-11-22 16:16:20 -08:00 |
|
|
Simo Lin
|
e01486778a
|
[model-gateway] update smg code owner (#13777)
|
2025-11-22 15:50:53 -08:00 |
|
|
Simo Lin
|
3ae664d786
|
[model-gateway] add both python and rust cli alias (#13678)
|
2025-11-20 17:00:28 -08:00 |
|
|
Simo Lin
|
6b9459e86c
|
[docker] fix dockerfile naming for diffusion (#13534)
|
2025-11-18 12:18:32 -08:00 |
|
|
Simo Lin
|
9188feccca
|
[model-gateway] use worker startup time out for worker registration (#13473)
|
2025-11-17 19:28:21 -08:00 |
|
|
Simo Lin
|
25acbbc612
|
Remove verbs from GET endpoint paths to follow REST standards (#13273)
|
2025-11-17 12:34:36 -08:00 |
|
|
Simo Lin
|
b436113fc2
|
[model-gateway] Add Gateway Release Tooling (#13420)
|
2025-11-17 03:26:48 -08:00 |
|
|
Simo Lin
|
b24235b8bb
|
[model-gateway] update workflow names for gateway and exclude npu (#13415)
|
2025-11-17 02:04:07 -08:00 |
|
|
Simo Lin
|
efc5d8f5ed
|
[model-gateway] fix SDist step readme path (#13373)
|
2025-11-16 03:16:52 -08:00 |
|
|
Simo Lin
|
ef32a252d0
|
[model-gateway] fix model gateway pypi release workflow path (#13372)
|
2025-11-16 02:33:49 -08:00 |
|
|
Simo Lin
|
37c8761567
|
[model-gateway] remove grpc feature flag and mark as default (#13330)
|
2025-11-15 03:16:57 -08:00 |
|
|
Simo Lin
|
b732ffa404
|
[model-gateway] move python to binding folder (#13295)
|
2025-11-15 00:32:21 -08:00 |
|
|
Simo Lin
|
172c71a23d
|
[model-gateway] smg release 0.2.3 (#13312)
|
2025-11-14 19:04:20 -08:00 |
|
|
Simo Lin
|
385ff0e56f
|
[model-gateway] change mg labeler from router to model-gateway (#13265)
|
2025-11-13 23:29:09 -08:00 |
|
|
Simo Lin
|
4c1e909a80
|
[router] minmax-m2 xml tool parser (#13148)
|
2025-11-12 23:24:54 -08:00 |
|
|
Simo Lin
|
6d21392b0e
|
[router] remove worker url requirement (#13172)
|
2025-11-12 17:32:58 -08:00 |
|
|
Simo Lin
|
3cdec20c6b
|
[router] add minmax m2 reasoning parser (#13137)
|
2025-11-12 18:27:05 +09:00 |
|
|
Simo Lin
|
ae68158f2e
|
[router] move radix tree to policy crate and addreses some code styles (#13131)
|
2025-11-11 23:36:04 -08:00 |
|
|
Simo Lin
|
6237754842
|
[router] 0.2.2 release (#12399)
|
2025-10-30 14:40:13 -07:00 |
|
|
Simo Lin
|
700daa3463
|
[router] harmony responses api streaming support (#12395)
|
2025-10-30 07:30:48 -07:00 |
|
|
Simo LinandChang Su
|
03b3e89aea
|
[router] Harmony Pipeline: Chat Completion & Responses API with MCP Support (#12153)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-10-29 18:52:56 -07:00 |
|
|
Simo Lin
|
5259becd3c
|
[bug] fix router installation to include additional dependency (#12348)
|
2025-10-29 12:45:18 -07:00 |
|
|
Simo Lin
|
b53d9e11c6
|
[bug] fix router pypi license file (#12345)
|
2025-10-29 10:05:00 -07:00 |
|
|
Simo Lin
|
84a9d0eab2
|
[router] support arm, windows, mac, linux, reduce wheel size and number (#12285)
|
2025-10-28 12:18:51 -07:00 |
|
|
Simo Lin
|
25e73640f4
|
[router] upgrade grpc dependency and py 3.13 3.14 support (#12284)
|
2025-10-28 08:51:32 -07:00 |
|
|
Simo Lin
|
d85d6dba3b
|
[router] configure workflow retries and timeout based on routerConfig (#12252)
|
2025-10-28 00:41:20 -07:00 |
|
|
Simo Lin
|
c5642a7a7a
|
[router] use mcp struct from sdk and clean up code across codebase (#12249)
|
2025-10-28 00:33:10 -07:00 |
|
|
Simo Lin
|
83b2240074
|
[router] remove code duplication (#12245)
|
2025-10-27 23:01:44 -07:00 |
|
|
Simo Lin
|
b89111d69b
|
add gitignore for claude code and serena mcp (#12166)
|
2025-10-26 13:30:17 -07:00 |
|
|
Simo Lin
|
9e949e5877
|
[router] centralize mcp tool args handling (#12155)
|
2025-10-26 01:47:12 -07:00 |
|
|
Simo Lin
|
03e7d94916
|
[router] update smg code owners for each component (#12141)
|
2025-10-25 22:28:59 -07:00 |
|
|
Simo Lin
|
212f5e4822
|
[router] MCP Manager Refactoring - Flat Architecture with Connection Pooling (#12097)
|
2025-10-25 21:26:28 -07:00 |
|
|
Simo Lin
|
22cbc9c086
|
[bugs] docker file name should be <name>.Dockerfile so it can properly render (#11869)
|
2025-10-25 08:34:43 -07:00 |
|
|
Simo Lin
|
229f236da8
|
[router] migrate app context to builder pattern 2/n (#12089)
|
2025-10-24 15:04:32 -07:00 |
|
|
Simo Lin
|
5983e5bd1b
|
[router] migrate app context to builder pattern 1/n (#12086)
|
2025-10-24 13:35:22 -07:00 |
|
|
Simo Lin
|
770d63123c
|
[router] fix ut router config init to use build pattern (#12084)
|
2025-10-24 11:16:31 -07:00 |
|
|
Simo LinandChang Su
|
dbf17a8313
|
[router] Add mTLS Support for Router-to-Worker Communication (#12019)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-10-23 23:14:09 -07:00 |
|
|
Simo Lin
|
6d6e24bcc4
|
[router] Add builder pattern for RouterConfig with zero duplication (#12030)
|
2025-10-23 16:46:10 -07:00 |
|
|
Simo Lin
|
a4b637d87a
|
[router] change ci names and update log level in ci (#12021)
|
2025-10-23 10:36:19 -07:00 |
|
|
Simo Lin
|
5dccf69713
|
[router] create worker removal step and clean up worker manager (#11921)
|
2025-10-22 13:26:06 -07:00 |
|
|
Simo Lin
|
8a801ee38d
|
[router] release router 0.2.1 (#11885)
|
2025-10-20 21:08:45 -07:00 |
|
|
Simo Lin
|
1111030395
|
[router] clean up workflow logs to debug for implementation details logs (#11886)
|
2025-10-20 18:24:55 -07:00 |
|
|
Simo Lin
|
b4948512b8
|
[router] remove encoding header for oai router (#11881)
|
2025-10-20 17:39:00 -07:00 |
|
|
Simo Lin
|
ddcba74b4d
|
[router] Worker Management Workflow Engine (#11868)
|
2025-10-20 17:00:22 -07:00 |
|
|
Simo Lin
|
a7ae61ed77
|
[router] Add Configurable L0 and L1 Tokenizer Caching (#11688)
|
2025-10-18 18:33:53 -07:00 |
|
|
Simo Lin
|
a5978a20f0
|
[router] fix grpc client time out to 1h (#11768)
|
2025-10-17 10:26:12 -07:00 |
|
|
Simo Lin
|
e483c1eae5
|
[router] Fix UTF-8 Boundary Panic in Stop Sequence Decoder (#11766)
|
2025-10-17 10:21:00 -07:00 |
|
|
Simo Lin
|
4f24ab1718
|
[router][grpc] add dissag info to warm up in grpc server (#11727)
|
2025-10-16 14:19:55 -07:00 |
|
|
Simo Lin
|
64affab495
|
[router] fix p and d worker filtering and bootstrap port handling (#11729)
|
2025-10-16 14:19:39 -07:00 |
|
|
Simo Lin
|
f5d30dae89
|
[router] Refactor StopSequenceDecoder to Use Sequence for Incremental Decoding (#11676)
|
2025-10-15 16:31:03 -07:00 |
|
|
Simo Lin
|
74737b2863
|
[router] upgrade to 0.2.0 (#11642)
|
2025-10-14 22:10:30 -04:00 |
|
|
Simo Lin
|
40e0082d8d
|
[router] add worker self discovery for metadata (#11638)
|
2025-10-14 22:07:25 -04:00 |
|
|
Simo Lin
|
e0c2af2ac2
|
[router] update router doc to latest features (#11639)
|
2025-10-14 18:32:30 -07:00 |
|
|
Simo LinandChang Su
|
325951460f
|
[router][grpc] add warm up to grpc server (#11627)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-10-14 16:11:16 -07:00 |
|
|
Simo Lin
|
49345a68cf
|
[router] update router readme to latest features (#11619)
|
2025-10-14 11:47:38 -07:00 |
|
|
Simo Lin
|
9e8a15a74c
|
[router] add chang and keyang to sgl router author (#11620)
|
2025-10-14 11:10:49 -07:00 |
|
|
Simo Lin
|
3962e39d7c
|
[router] cleanup app context and move to startup (#11617)
|
2025-10-14 10:19:28 -07:00 |
|
|
Simo Lin
|
a04efc4933
|
[router] when given both local tokenizer and chat template, log all (#11601)
|
2025-10-14 02:22:58 -07:00 |
|
|
Simo Lin
|
da7fac1b75
|
[router] allow router launch server to use grpc mode (#11600)
|
2025-10-14 01:42:43 -07:00 |
|
|
Simo Lin
|
28ad2297a0
|
[router] delete useless table content comment in spec (#11597)
|
2025-10-14 01:08:18 -07:00 |
|
|
Simo Lin
|
4b62af92ef
|
[router] change worker api to async instead of sync (#11566)
|
2025-10-14 00:32:21 -07:00 |
|
|
Simo Lin
|
0b9915c132
|
[router] update generate spec to align with sgl io struct (#11591)
|
2025-10-14 02:51:33 -04:00 |
|
|
Simo Lin
|
728af88781
|
[router] allow user to specify chat template path (#11549)
|
2025-10-13 10:47:57 -07:00 |
|
|
Simo Lin
|
7c94eaeeb0
|
[router] allow tokenizer path to be dir (#11530)
|
2025-10-13 09:30:09 -04:00 |
|
|
Simo Lin
|
13d596c93e
|
[router][ci] Add Nightly Release Workflow for SGLang Router (#11527)
|
2025-10-13 09:28:55 -04:00 |
|
|
Simo Lin
|
c495833186
|
[router] leverage RAII to actively cancel request during client disconnect (#11399)
|
2025-10-10 20:43:38 -04:00 |
|
|
Simo Lin
|
2eeb27515a
|
[router] disable rate limiter by default (#11435)
|
2025-10-10 20:43:07 -04:00 |
|
|
Simo Lin
|
88bb627d0d
|
[router] change grpc client from mutable to clone (#11394)
|
2025-10-09 11:00:24 -07:00 |
|
|
Simo Lin
|
368fd20622
|
[router][grpc] disable health check generation and increase timeout (#11353)
|
2025-10-08 19:23:08 -07:00 |
|
|
Simo Lin
|
677aa0e25f
|
[router] improve reasoning parser lock and reduce req cloning (#11336)
|
2025-10-08 11:18:15 -07:00 |
|
|
Simo Lin
|
01c9ee1ab4
|
[router] refactor generate to use new pipeline arch (#11323)
|
2025-10-08 09:38:50 -07:00 |
|
|
Simo Lin
|
fde9b96392
|
[router] cleanup worker health check to return early (#11310)
|
2025-10-07 16:53:10 -07:00 |
|
|
Simo Lin
|
f4affd4df5
|
[router] fix grpc connection conversion and add optimization (#11305)
|
2025-10-07 10:39:33 -07:00 |
|
|
Simo Lin
|
2fcd56eaf6
|
[router] add get server info and get model info in grpc server (#11303)
|
2025-10-07 08:36:52 -07:00 |
|
|
Simo Lin
|
79d3495177
|
[router] add reasoning and tool parser argument in router (#11290)
|
2025-10-07 09:08:32 -04:00 |
|
|
Simo Lin
|
5ee777c98f
|
[router] add ipv6 support across all components (#11219)
|
2025-10-06 08:16:59 -07:00 |
|
|
Simo Lin
|
d736e0b65e
|
[router] add grpc router pd mode for chat and generate (#11140)
|
2025-10-04 06:58:28 -07:00 |
|
|
Simo Lin
|
ffd03a9bd3
|
[router] fix get load response parsing (#11213)
|
2025-10-04 06:58:02 -07:00 |
|
|
Simo Lin
|
96fe2d0f15
|
[router] add pd service in grpc router for pd (#11120)
|
2025-10-01 08:09:21 -07:00 |
|
|
Simo Lin
|
5a290a5644
|
[router][grpc-server] Fix gRPC server shutdown (#11094)
|
2025-09-30 07:12:12 -04:00 |
|
|
Simo LinandChang Su
|
33b3c0f85f
|
[router] grpc router generate endpoint support (#11070)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-09-29 22:07:53 -07:00 |
|
|
Simo Lin
|
816b3a433a
|
[router] add n to generate sampling params (#11069)
|
2025-09-29 07:37:43 -07:00 |
|
|
Simo Lin
|
2572886367
|
[router] add harmony tool parser base structure and interface (#11036)
|
2025-09-28 19:46:38 -07:00 |
|
|
Simo Lin
|
336e9a6058
|
[router] migrate to rust python module for pythonic parser (#11033)
|
2025-09-28 14:48:59 -04:00 |
|
|
Simo Lin
|
5519766a4d
|
[router] fix chat template loading and tokenizer path (#10999)
|
2025-09-27 23:54:12 -04:00 |
|
|
Simo Lin
|
aae7ead2d0
|
[router] remove old/oudated/useless comments across code base (#10968)
|
2025-09-26 10:48:50 -07:00 |
|
|
Simo Lin
|
a7fe6e10a1
|
[router] remove old/oudated/useless comments (#10967)
|
2025-09-26 09:45:15 -07:00 |
|
|
Simo Lin
|
be059b83d6
|
[router] grpc router regular mode import cleanup (#10963)
|
2025-09-26 04:06:59 -07:00 |
|
|
Simo Lin
|
5d4fe1ceee
|
[router] add move grpc worker management from router to worker manager (#10960)
|
2025-09-26 03:57:57 -07:00 |
|