Commit Graph

241 Commits

Author SHA1 Message Date
Simo Lin
c8dc4d2d1f [smg][ci] migrate enable_thinking tests to new infrastructure (#16739) 2026-01-08 06:57:45 -08:00
Simo Lin
f52ae586b6 Remove migrated e2e_grpc/basic tests (#16738) 2026-01-08 06:30:45 -08:00
Simo Lin
2f8a36347c [smg][ci] migrate chat completions tests to new infrastructure and build wheel once and share via artifact (#16709) 2026-01-08 06:29:23 -08:00
Simo Lin
4c46ecde80 [smg][ci] delete old responses api ci (#16695) 2026-01-07 18:24:12 -08:00
Simo Lin
a08dc5aa10 [smg][ci] rename 3rd models from cloud backend and delete dead code (#16692) 2026-01-07 18:19:44 -08:00
Simo Lin
109fe03ad1 [smg][ci] Migrate Response API e2e tests to shared infrastructure (#16680)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: key4ng <rukeyang@gmail.com>
2026-01-07 17:40:03 -08:00
Simo Lin
6037267f5b [smg][ci] Add thread safety to ModelPool and GPUAllocator (#16674) 2026-01-07 13:25:41 -08:00
Simo Lin
7385834c8d Add reference counting to ModelInstance for parallel test safety (#16672) 2026-01-07 08:28:15 -08:00
Ziwen Zhao
b5a94f8a8e [model-gateway] Fix IGW routing for external OpenAI workers (#16633) 2026-01-07 08:15:23 -08:00
Simo Lin
c356ed03dd refactor(e2e): unify RouterInstance into Gateway class, split conftest.py into modular fixtures (#16671) 2026-01-07 07:50:28 -08:00
Simo Lin
55b7936582 refactor(e2e_test): fix smg ci e2e test code quality (#16664) 2026-01-07 06:59:49 -08:00
Simo Lin
8729ad5e6c fix(e2e_test): remove dead code and fix type annotations (#16661) 2026-01-07 06:35:15 -08:00
Simo Lin
e432057381 [smg][ci] preserve model launch order with test collected (#16618) 2026-01-07 06:16:59 -08:00
fzyzcjy
3be1e734ee [model-gateway] extract header extraction in policy and add (#16566) 2026-01-06 20:18:44 -08:00
Simo Lin
d8b8198192 [smg][ci]: migrate benchmarks to e2e_test/benchmarks/, use parent conftest (#16597) 2026-01-06 20:15:20 -08:00
Chang Su
fb5b71d015 [router][openai] Rename prepare_mcp_payload_for_streaming and patch_streaming_response_json (#16596) 2026-01-06 16:33:29 -08:00
Chang Su
05b54b6d7b [router][grpc] Replace Vec<(String, String, String)> with ExtractedToolCall (#16598) 2026-01-06 16:32:59 -08:00
Chang Su
4f443f445a [model-gateway][cleanup] Fix wrong comment in manager.rs (#16601) 2026-01-06 16:32:37 -08:00
Simo Lin
dce8b0606c refactor(e2e): keep only benchmark tests in e2e_http, remove redundant tests (#16594) 2026-01-06 15:05:08 -08:00
Simo Lin
d415d22daa refactor(e2e): remove old embedding tests migrated to e2e_test/embeddings (#16592) 2026-01-06 14:07:45 -08:00
Simo Lin
a49b9a6420 [model-gateway] add embedding tests (#16583) 2026-01-06 13:24:37 -08:00
Simo Lin
d57d8e7e5f [smg] clean up logs in mcp (should be info instead warn) (#16591) 2026-01-06 12:44:32 -08:00
Simo Lin
a3656cbb96 [ci] fix url strips in smg ci (#16548)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
2026-01-06 09:23:58 -08:00
Chang Su
21da2dc1ce [grpc] Unify ResponsesContext and HarmonyResponsesContext (#16549) 2026-01-06 07:28:06 -08:00
Chang Su
9bf76c11a9 [responses API] Add list_tools_for_servers and threading server_keys in routers (#16540) 2026-01-05 22:42:22 -08:00
Chang Su
4cf2bbd084 [router] Remove deadcode and add note for unused API completeness methods (#16528) 2026-01-05 19:53:28 -08:00
Simo Lin
402a0bd6dc [model-gateway] Add model scope support and LRU eviction for GPU-constrained environments (#16525) 2026-01-05 18:28:07 -08:00
Chang Su
b98bd9a5fb [model-gateway] Tighten visibility in modules and remove unused re-exports (#16524) 2026-01-05 17:38:50 -08:00
Simo Lin
4221b7c573 [model-gateway] refactor e2e test infrastructure and add router CI (#16513) 2026-01-05 13:50:59 -08:00
Chang Su
1751c75b5d [model-gateway] Tighten visibility across data_connector and grpc module (#16516) 2026-01-05 12:31:32 -08:00
realWeilai
23849eba7b [model-gateway] fix tokenizer encode in golang bindings (#16482) 2026-01-05 11:40:09 -08:00
Chang Su
51541404f8 [grpc] Refactor openai module (#16511) 2026-01-05 11:39:33 -08:00
Chang Su
5a2b1ed407 [grpc] Refactor grpc/regular/responses (#16509) 2026-01-05 10:21:34 -08:00
Chang Su
454dc9e242 [model-gateway][grpc] Refactor harmony/responses.rs (#16508) 2026-01-05 10:21:05 -08:00
fzyzcjy
1e41069ad1 Fix age bucket rendering issue (#16492) 2026-01-05 09:29:14 -08:00
Simo Lin
2b4d6d813b [model-gateway][e2e_test]: Create directory structure and backends config (#16469) 2026-01-05 08:01:53 -08:00
Praneth Paruchuri
b12258bfaa [model-gateway] Optimize HTTP Router Fan-out: Replace Serial Execution with Concurrent Streams (#16042) 2026-01-04 23:38:45 -08:00
Simo Lin
7f6a678f8f [model-gateway] add GPU allocator and model pool infrastructure for parallel E2E tests (#16460) 2026-01-04 23:30:32 -08:00
Simo Lin
e6fe092dcc [model-gateway] rename py_test to e2e_test (#16454) 2026-01-04 22:24:34 -08:00
Ratish P
f02d82211a [model-gateway]: move PD configuration conflict checks to model gateway (#16088) 2026-01-05 14:16:07 +08:00
Simo Lin
e53160bb31 [model-gateway] reorganize integration tests into logical subdirectories (#16451) 2026-01-04 21:49:20 -08:00
Simo Lin
2181bc9e51 [model-gateway] Delete Python integration_mock tests (#16448) 2026-01-04 21:34:06 -08:00
Simo Lin
f84487af59 [model-gateway] : Rust integration tests for integration_mock replacement (#16441) 2026-01-04 21:01:47 -08:00
Simo Lin
0ff3747ca1 [model-gateway]: move unit tests to bindings/python/tests/ (#16430) 2026-01-04 18:10:35 -08:00
Simo Lin
4436dc0f6c [model-gateway] improve lock contention and allocation in middleware (#16405) 2026-01-04 10:40:07 -08:00
Simo Lin
cf6800f6cc [model-gateway] optimize Vec and HashMap allocations in responses api (#16406) 2026-01-04 10:07:55 -08:00
fzyzcjy
f16606d6f6 Super tiny code cleanup (#16401) 2026-01-04 23:00:09 +08:00
fzyzcjy
27a0830593 Tiny add manual policy benchmark to CI (#16392) 2026-01-04 19:10:35 +08:00
Chang Su
8d58b3dcdd [model-gateway] Clear architectual debt in responses API (#16359) 2026-01-03 21:35:28 -08:00
fzyzcjy
216ea910ad Tiny support forwarding SMG routing key to engine for dumping and avoid string allocation (#16352) 2026-01-04 08:33:35 +08:00