 Simo Lin
|
a16f03c1e3
|
Revert "feat: add banner to sgl-model-gateway (#20471)" (#20536)
|
2026-03-13 09:20:32 -07:00 |
|
|
Simo Lin
|
654fc02cf1
|
[gRPC] Extract gRPC servicer into standalone package (#20478)
Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
|
2026-03-13 09:13:29 -07:00 |
|
|
Simo Lin
|
b227e53ebf
|
feat: add banner to sgl-model-gateway (#20471)
Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
|
2026-03-12 14:25:27 -07:00 |
|
|
Simo LinandChang Su
|
3f3eb206fa
|
feat(grpc): add SubscribeKvEvents RPC for KV cache event streaming (#20112)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2026-03-08 16:00:29 -07:00 |
|
|
Simo Lin
|
9502369488
|
fix(grpc): add server-side keepalive options to prevent GOAWAY (#19986)
Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
|
2026-03-05 18:56:35 -08:00 |
|
|
Simo Lin
|
b2166a2f33
|
fix(gateway): pin internal crate dependencies to exact versions (#19224)
Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
|
2026-02-23 22:35:47 -08:00 |
|
|
Simo Lin
|
88a7e48108
|
fix(grpc): handle embedding requests in _handle_batch_output (#19221)
Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
|
2026-02-23 22:32:16 -08:00 |
|
|
Simo Lin
|
edba96b98a
|
fix(docker): migrate ROCm Dockerfiles from setuptools-rust to maturin (#19210)
Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
|
2026-02-23 19:00:05 -08:00 |
|
|
Simo LinandChang Su
|
bf08d3f43c
|
[gRPC] Fix scheduler startup broken by context parallel refactor (#18933)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2026-02-17 08:52:11 -08:00 |
|
|
Simo Lin
|
0236522214
|
refactor: remove crate re-export aliases from lib.rs (#18737)
|
2026-02-12 09:17:24 -08:00 |
|
|
Simo Lin
|
2f283d8152
|
refactor: consolidate gRPC client into shared crate dependency (#18730)
|
2026-02-12 06:43:24 -08:00 |
|
|
Simo Lin
|
92c5749f41
|
refactor: replace local proto compilation with smg-grpc-proto package (#18682)
|
2026-02-12 05:29:24 -08:00 |
|
|
Simo Lin
|
ac16d4450e
|
[smg][mesh] extract mesh to mesh crate to reduce compile time (#17907)
|
2026-01-28 17:58:42 -08:00 |
|
|
Simo Lin
|
6756cf18c6
|
remove multimodal as this is completely dead code (#17750)
|
2026-01-26 00:07:40 -08:00 |
|
|
Simo Lin
|
d4adff31aa
|
update wasm endpoint (#17748)
|
2026-01-25 23:43:29 -08:00 |
|
|
Simo Lin
|
ed75136e85
|
remove self managed wasm as it has been replaced with official smg wa… (#17746)
|
2026-01-25 22:56:33 -08:00 |
|
|
Simo Lin
|
7890a96f97
|
remove self managed mcp as it has been replaced with official rmcp crate (#17740)
|
2026-01-25 22:17:06 -08:00 |
|
|
Simo Lin
|
46bc53a81b
|
[smg] import db crate to replace self managed one (#17727)
|
2026-01-25 16:54:52 -08:00 |
|
|
Simo Lin
|
52d0ca944b
|
[smg] import official wfaas crate to replace self managed one (#17724)
|
2026-01-25 15:20:58 -08:00 |
|
|
Simo Lin
|
bf139af37a
|
[smg] use already published auth crate for better compilation speed (#17723)
|
2026-01-25 14:52:42 -08:00 |
|
|
Simo Lin
|
97a36a72b7
|
[smg] remove dead tokenizer code (#17722)
|
2026-01-25 14:04:48 -08:00 |
|
|
Simo Lin
|
f5ac1ca10b
|
[smg] use official tokenizer crate instead of manually built one (#17721)
|
2026-01-25 13:35:16 -08:00 |
|
|
Simo Lin
|
2c1e2674cc
|
[misc] replace existing tool call code with new crate package (#17720)
|
2026-01-25 13:07:39 -08:00 |
|
|
Simo Lin
|
48f5c46a8d
|
[misc] remove tool parser and tree benchmark as they are not meaningful atm (#17719)
|
2026-01-25 13:03:06 -08:00 |
|
|
Simo Lin
|
86c7bc6431
|
[smg] update crate for tools (#17717)
|
2026-01-25 12:56:42 -08:00 |
|
|
Simo Lin
|
7a1d7ab47b
|
remove self managed protocols as it has been replaced with official oai spec (#17711)
|
2026-01-25 06:38:13 -08:00 |
|
|
Simo Lin
|
8db2802b2d
|
update to use official openai protocol crate (#17710)
|
2026-01-25 06:35:59 -08:00 |
|
|
Simo Lin
|
6af22f8dbf
|
use published reasoning parser crate (#17709)
|
2026-01-25 06:15:03 -08:00 |
|
|
Simo Lin
|
3f44268fe5
|
[smg] release 0.3.2 (#17168)
|
2026-01-15 11:22:18 -08:00 |
|
|
Simo Lin
|
f7ec8174db
|
[smg][ci] add make cmd to patch versions (#17167)
|
2026-01-15 11:00:53 -08:00 |
|
|
Simo Lin
|
f091858304
|
[gRPC] Add GetLoads RPC for comprehensive load metrics (#17087)
|
2026-01-14 12:05:10 -08:00 |
|
|
Simo Lin
|
b3a3f51320
|
[API] Add /v1/loads endpoint for load metrics (#16976)
|
2026-01-14 09:13:20 -08:00 |
|
|
Simo Lin
|
c6a64e9f69
|
[smg] fix type complexity for workflow run_if (#16981)
|
2026-01-12 15:41:52 -08:00 |
|
|
Simo Lin
|
e0ac559ae1
|
feat(workflow): add scheduled/delayed steps and conditional branching (#16980)
|
2026-01-12 15:06:12 -08:00 |
|
|
Simo Lin
|
6620548fd8
|
[model-gateway] make StateStore trait async for external persistence (#16979)
|
2026-01-12 12:55:59 -08:00 |
|
|
Simo Lin
|
6e158e55b4
|
[model-gateway] improve workflow engine code quality (#16977)
|
2026-01-12 12:14:46 -08:00 |
|
|
Simo Lin
|
ed729d22b3
|
[model-gateway] refactor workflow engine from type erasure to typed engines (#16973)
|
2026-01-12 10:47:00 -08:00 |
|
|
Simo Lin
|
fa51b85466
|
[model-gateway] convert workflow system to type-safe workflow data (#16970)
|
2026-01-12 10:06:36 -08:00 |
|
|
Simo Lin
|
d0092decb1
|
[model-gateway] Fix workflow engine race conditions and add graceful shutdown (#16963)
|
2026-01-12 06:37:39 -08:00 |
|
|
Simo Lin
|
7c25687c9b
|
fix(gateway): rewrite gauge_histogram.rs for zero-allocation hot path (#16878)
|
2026-01-10 18:15:21 -08:00 |
|
|
Simo Lin
|
7460240737
|
[model-gateway] release 0.3.1 (#16254)
|
2026-01-08 17:49:37 -08:00 |
|
|
Simo LinandChang Su
|
1bc7aa5801
|
[smg] update gRPC proto to match upstream changes (#16764)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2026-01-08 16:33:09 -08:00 |
|
|
Simo Lin
|
8726d30cb2
|
[smg] Add Nemotron Nano V3 reasoning parser support (#16763)
|
2026-01-08 16:29:24 -08:00 |
|
|
Simo Lin
|
ee71e773e1
|
[smg] Work around sglang's notorious orphan process problem (#16756)
|
2026-01-08 14:41:03 -08:00 |
|
|
Simo Lin
|
55a8dd0095
|
[grpc] Fix protobuf compilation in isolated build environments (#16754)
|
2026-01-08 13:09:01 -08:00 |
|
|
Simo Lin
|
cf24232100
|
fix(e2e): prevent potential hangs in model pool subprocess handling (#16752)
|
2026-01-08 12:35:56 -08:00 |
|
|
Simo Lin
|
9d03af916a
|
[smg][ci] delete old chat completion integration tests and workflow step (#16751)
|
2026-01-08 11:31:54 -08:00 |
|
|
Simo Lin
|
49305fa1c1
|
[smg][ci] migrate function calling tests to new infrastructure (#16748)
|
2026-01-08 11:08:01 -08:00 |
|
|
Simo Lin
|
fbc24886dd
|
[smg][ci] migrate validation tests to new infrastructure (#16746)
|
2026-01-08 09:26:31 -08:00 |
|
|
Simo Lin
|
8a45a9c6a9
|
[smg][ci] fix model pool GPU cleanup and add startup reliability improvements (#16745)
|
2026-01-08 08:55:18 -08:00 |
|
|
Simo Lin
|
aecd5f5f3e
|
[smg][ci] migrate reasoning_content tests to new infrastructure (#16741)
|
2026-01-08 08:25:39 -08:00 |
|
|
Simo Lin
|
c8dc4d2d1f
|
[smg][ci] migrate enable_thinking tests to new infrastructure (#16739)
|
2026-01-08 06:57:45 -08:00 |
|
|
Simo Lin
|
f52ae586b6
|
Remove migrated e2e_grpc/basic tests (#16738)
|
2026-01-08 06:30:45 -08:00 |
|
|
Simo Lin
|
2f8a36347c
|
[smg][ci] migrate chat completions tests to new infrastructure and build wheel once and share via artifact (#16709)
|
2026-01-08 06:29:23 -08:00 |
|
|
Simo Lin
|
4c46ecde80
|
[smg][ci] delete old responses api ci (#16695)
|
2026-01-07 18:24:12 -08:00 |
|
|
Simo Lin
|
a08dc5aa10
|
[smg][ci] rename 3rd models from cloud backend and delete dead code (#16692)
|
2026-01-07 18:19:44 -08:00 |
|
|
|
109fe03ad1
|
[smg][ci] Migrate Response API e2e tests to shared infrastructure (#16680)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: key4ng <rukeyang@gmail.com>
|
2026-01-07 17:40:03 -08:00 |
|
|
Simo Lin
|
6037267f5b
|
[smg][ci] Add thread safety to ModelPool and GPUAllocator (#16674)
|
2026-01-07 13:25:41 -08:00 |
|
|
Simo Lin
|
7385834c8d
|
Add reference counting to ModelInstance for parallel test safety (#16672)
|
2026-01-07 08:28:15 -08:00 |
|
|
Simo Lin
|
c356ed03dd
|
refactor(e2e): unify RouterInstance into Gateway class, split conftest.py into modular fixtures (#16671)
|
2026-01-07 07:50:28 -08:00 |
|
|
Simo Lin
|
55b7936582
|
refactor(e2e_test): fix smg ci e2e test code quality (#16664)
|
2026-01-07 06:59:49 -08:00 |
|
|
Simo Lin
|
8729ad5e6c
|
fix(e2e_test): remove dead code and fix type annotations (#16661)
|
2026-01-07 06:35:15 -08:00 |
|
|
Simo Lin
|
e432057381
|
[smg][ci] preserve model launch order with test collected (#16618)
|
2026-01-07 06:16:59 -08:00 |
|
|
Simo Lin
|
d8b8198192
|
[smg][ci]: migrate benchmarks to e2e_test/benchmarks/, use parent conftest (#16597)
|
2026-01-06 20:15:20 -08:00 |
|
|
Simo Lin
|
dce8b0606c
|
refactor(e2e): keep only benchmark tests in e2e_http, remove redundant tests (#16594)
|
2026-01-06 15:05:08 -08:00 |
|
|
Simo Lin
|
d415d22daa
|
refactor(e2e): remove old embedding tests migrated to e2e_test/embeddings (#16592)
|
2026-01-06 14:07:45 -08:00 |
|
|
Simo Lin
|
a49b9a6420
|
[model-gateway] add embedding tests (#16583)
|
2026-01-06 13:24:37 -08:00 |
|
|
Simo Lin
|
d57d8e7e5f
|
[smg] clean up logs in mcp (should be info instead warn) (#16591)
|
2026-01-06 12:44:32 -08:00 |
|
|
Simo LinandChang Su
|
a3656cbb96
|
[ci] fix url strips in smg ci (#16548)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2026-01-06 09:23:58 -08:00 |
|
|
Simo Lin
|
402a0bd6dc
|
[model-gateway] Add model scope support and LRU eviction for GPU-constrained environments (#16525)
|
2026-01-05 18:28:07 -08:00 |
|
|
Simo Lin
|
4221b7c573
|
[model-gateway] refactor e2e test infrastructure and add router CI (#16513)
|
2026-01-05 13:50:59 -08:00 |
|
|
Simo Lin
|
2b4d6d813b
|
[model-gateway][e2e_test]: Create directory structure and backends config (#16469)
|
2026-01-05 08:01:53 -08:00 |
|
|
Simo Lin
|
7f6a678f8f
|
[model-gateway] add GPU allocator and model pool infrastructure for parallel E2E tests (#16460)
|
2026-01-04 23:30:32 -08:00 |
|
|
Simo Lin
|
399ca037b1
|
[bug] fix lint blocking ci (#16464)
|
2026-01-04 22:57:33 -08:00 |
|
|
Simo Lin
|
e6fe092dcc
|
[model-gateway] rename py_test to e2e_test (#16454)
|
2026-01-04 22:24:34 -08:00 |
|
|
Simo Lin
|
e53160bb31
|
[model-gateway] reorganize integration tests into logical subdirectories (#16451)
|
2026-01-04 21:49:20 -08:00 |
|
|
Simo Lin
|
2181bc9e51
|
[model-gateway] Delete Python integration_mock tests (#16448)
|
2026-01-04 21:34:06 -08:00 |
|
|
Simo Lin
|
f84487af59
|
[model-gateway] : Rust integration tests for integration_mock replacement (#16441)
|
2026-01-04 21:01:47 -08:00 |
|
|
Simo Lin
|
0ff3747ca1
|
[model-gateway]: move unit tests to bindings/python/tests/ (#16430)
|
2026-01-04 18:10:35 -08:00 |
|
|
Simo Lin
|
4436dc0f6c
|
[model-gateway] improve lock contention and allocation in middleware (#16405)
|
2026-01-04 10:40:07 -08:00 |
|
|
Simo Lin
|
cf6800f6cc
|
[model-gateway] optimize Vec and HashMap allocations in responses api (#16406)
|
2026-01-04 10:07:55 -08:00 |
|
|
Simo Lin
|
65b0b5b248
|
fix(logging): use Display format for model_id instead of Debug (#16337)
|
2026-01-03 00:21:39 -08:00 |
|
|
Simo Lin
|
31ed68e7c1
|
[model-gateway] address embedding similarity threshold in ut (#16321)
|
2026-01-02 16:46:55 -08:00 |
|
|
Simo LinandChang Su
|
d401d23876
|
[model-gateway] Add embedding correctness test comparing against HuggingFace (#16092)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2026-01-02 10:58:20 -08:00 |
|
|
Simo Lin
|
851845579c
|
perf(tree): optimize input_char_count calculation (#16238)
|
2025-12-31 15:29:37 -08:00 |
|
|
Simo Lin
|
ace6f3005f
|
ci(benchmark): disable sccache summary annotations and fix tree output (#16233)
|
2025-12-31 11:05:21 -08:00 |
|
|
Simo Lin
|
60d7279c46
|
test(tree): add comprehensive unit tests and fix input_char_count bug (#16228)
|
2025-12-31 07:31:39 -08:00 |
|
|
Simo Lin
|
b4ce7a6d71
|
[model-gateway] cache_aware eliminate String allocations in hot path (#16209)
|
2025-12-31 00:26:10 -08:00 |
|
|
Simo Lin
|
9940c6f592
|
[model-gateway]: ASCII byte comparison and probabilistic timestamp updates (#16181)
|
2025-12-30 12:28:01 -08:00 |
|
|
Simo LinandChang Su
|
4bc2f2e0bc
|
[model-gateway][docs] Add Classification API documentation (#16182)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-30 10:53:31 -08:00 |
|
|
Simo Lin
|
8c6f865afb
|
[model-gateway]: optimize prefix_match with zero-copy tenant and deferred char count (#16099)
|
2025-12-29 11:32:57 -08:00 |
|
|
Simo LinandChang Su
|
f39382c6ca
|
[model-gateway] Fix duplicate classify prefix in response ID (#16101)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 11:07:21 -08:00 |
|
|
Simo LinandChang Su
|
2ff289e2dc
|
[model-gateway] Generate UUID-based request IDs for embedding/classify (#16100)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 10:49:58 -08:00 |
|
|
Simo LinandChang Su
|
b2a3f0551a
|
[model-gateway] Wire classify pipeline to gRPC router (#16098)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 10:44:45 -08:00 |
|
|
Simo Lin
|
684e148e38
|
[model-gateway] Optimize INSERT with leaf-only timestamp updates (#16097)
|
2025-12-29 10:43:20 -08:00 |
|
|
Simo LinandChang Su
|
f44c4b3715
|
[model-gateway] Add classify pipeline stages and protocol types (#16094)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 10:23:36 -08:00 |
|
|
Simo Lin
|
ac78f96ec5
|
[model-gateway] Optimize radix tree timestamp updates for multi-tenant scaling (#16093)
|
2025-12-29 10:08:19 -08:00 |
|
|
Simo Lin
|
7f5879985c
|
[model-gateway] Improve tree benchmark with realistic multi-tenant scenarios (#14838)
|
2025-12-29 08:37:24 -08:00 |
|
|
Simo LinandChang Su
|
162d1cf9be
|
[model-gateway] Add classification model support infrastructure (#16061)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 08:34:05 -08:00 |
|
|
Simo Lin
|
c31f62722c
|
[model-gateway] fix tokenizer to match transformers special token handling (#16087)
|
2025-12-29 08:13:03 -08:00 |
|