Commit Graph

9885 Commits

Author SHA1 Message Date
Simo Lin
f84487af59 [model-gateway] : Rust integration tests for integration_mock replacement (#16441) 2026-01-04 21:01:47 -08:00
Huapeng Zhou
078270473a [Doc] Default lora backend: csgmv (#16444) 2026-01-05 12:45:49 +08:00
fzyzcjy
c63e9cb29e Super tiny fix CI (#16437) 2026-01-04 20:34:49 -08:00
YAMY
12cde0df99 [SPEC_V2] Fix Acclen drop when enabling DP Attention for Spec-Overlap (#16310) 2026-01-05 11:39:23 +08:00
Liangsheng Yin
a7fd810842 Allow editable install without .git with add fallback version in pyproject.toml (#16435) 2026-01-05 11:17:20 +08:00
Kangyan-Zhou
ca80c19b55 Revert "[diffusion] feat: support warmup with resolutions" (#16433) 2026-01-04 18:44:05 -08:00
fzyzcjy
1e7b326482 Super tiny fix main code (#16432) 2026-01-05 10:34:25 +08:00
CHEN Xi
e267ca0beb [diffusion] doc: document LoRA support in CLI (#16375) 2026-01-05 10:20:30 +08:00
Mick
9a8ba3c189 [diffusion] feat: support warmup with resolutions (#16330) 2026-01-05 10:16:26 +08:00
Xiaoyu Zhang
0fee6bc632 [JIT kernel] Apply jit per_tensor_quant_fp8 kernel (#15836) 2026-01-05 10:15:00 +08:00
Simo Lin
0ff3747ca1 [model-gateway]: move unit tests to bindings/python/tests/ (#16430) 2026-01-04 18:10:35 -08:00
Alison Shao
f8411ded6e ci: migrate 1-GPU model tests to test/registered/models/ (#16414) 2026-01-04 18:08:01 -08:00
fzyzcjy
249c356331 Super tiny update tokenizer benchmark (#16429) 2026-01-05 09:14:52 +08:00
fzyzcjy
12df16607b Tiny speed up kimi detokenizer by 10x (#16427) 2026-01-05 09:12:05 +08:00
Douglas Yang
55d112dc79 fix: enable multi-threading for h200 tests (#16413) 2026-01-04 14:43:53 -08:00
Douglas Yang
a1ed247fd7 feature: add runner online count to failure monitor (#16408) 2026-01-04 13:24:04 -08:00
Douglas Yang
87699d48eb fix: only publish trace from tp 0 (#16411) 2026-01-04 12:17:08 -08:00
Alison Shao
52c604342c chore: print test list at beginning and end of run_suite.py (#16334)
Co-authored-by: Mick <mickjagger19@icloud.com>
2026-01-04 11:52:52 -08:00
Alison Shao
ff0f370f85 ci: migrate MoE tests to test/registered/moe/ (#16127) 2026-01-04 11:51:12 -08:00
Alison Shao
26f9e20755 ci: migrate quantization kernel tests to test/registered/quant/ (#16323) 2026-01-04 11:49:29 -08:00
Yingchun Lai
828cd8936f Introduce sgl-kernel Dockerfile (#14066) 2026-01-04 11:19:08 -08:00
Simo Lin
4436dc0f6c [model-gateway] improve lock contention and allocation in middleware (#16405) 2026-01-04 10:40:07 -08:00
Simo Lin
cf6800f6cc [model-gateway] optimize Vec and HashMap allocations in responses api (#16406) 2026-01-04 10:07:55 -08:00
fzyzcjy
f16606d6f6 Super tiny code cleanup (#16401) 2026-01-04 23:00:09 +08:00
fzyzcjy
387fad2f74 Tiny add detokenization benchmarks (#16400) 2026-01-04 22:53:38 +08:00
Ke Bao
76bc07a335 Move swa memory pool to a seperate file (#16347) 2026-01-04 22:39:30 +08:00
Ke Bao
b328cd20bb Skip local attn init metadata for mimo swa model (#16349) 2026-01-04 22:38:36 +08:00
Hexq0210
bf32cd8397 [npu] update model and feature supported for ascend npu (#16390) 2026-01-04 21:50:20 +08:00
fzyzcjy
27a0830593 Tiny add manual policy benchmark to CI (#16392) 2026-01-04 19:10:35 +08:00
fzyzcjy
53479e226b Tiny refactor router benchmark github actions (#16374) 2026-01-04 18:00:12 +08:00
Lianmin Zheng
ff978e7d4c Update readme (#16391) 2026-01-04 01:57:54 -08:00
Douglas Yang
16e00651dd fix: health check adding test delays (#16381) 2026-01-04 00:28:32 -08:00
fzyzcjy
1b2b95d8d1 Super tiny cleanup unused function (#16376) 2026-01-04 15:08:12 +08:00
Shangming Cai
9cac3c8628 Add liusy58 into CI_PERMISSION (#16373)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
2026-01-03 21:36:35 -08:00
Chang Su
8d58b3dcdd [model-gateway] Clear architectual debt in responses API (#16359) 2026-01-03 21:35:28 -08:00
Yuan Luo
5f3eb377e0 [VLM] Support request level max_dynamic_patch for OpenAI request (#16268)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-01-04 13:04:43 +08:00
Liangsheng Yin
229938805f Make personal configs optional in SGLang's official docker image. (#16365) 2026-01-04 12:27:13 +08:00
fzyzcjy
d7aa0ce72f Support multiple engines for router simulation in schedule simulator (#16368) 2026-01-04 11:46:05 +08:00
fzyzcjy
e797f0c570 Support offline generation scenario for prefill delayer (#16363) 2026-01-04 11:05:10 +08:00
fzyzcjy
5d4b7c78bf Fix memory leak in prefill delayer (#16358) 2026-01-04 10:11:18 +08:00
Yuan Luo
9bd64d739b [VLM] Add doc for ViT CUDA Graph (#16343)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-01-04 10:05:23 +08:00
fzyzcjy
24e116ef6a Enhance schedule simulator and add several e2e scenarios (#16357) 2026-01-04 10:02:38 +08:00
fzyzcjy
8ca9597057 Tiny support sticky routing algorithm in schedule simulator (#16355) 2026-01-04 08:36:52 +08:00
fzyzcjy
216ea910ad Tiny support forwarding SMG routing key to engine for dumping and avoid string allocation (#16352) 2026-01-04 08:33:35 +08:00
Hudson Xing
f4ab2ec5be Add unified metrics collection framework (v1) (#16064)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-01-03 16:30:38 -08:00
fy
25fa2ac290 Convert cu_seqlens to CPU for npu_flash_attention_unpad operator (#15434) 2026-01-04 08:16:42 +08:00
fzyzcjy
ef5ac6f01e Support shared prefix (gsp) in schedule simulator (#16353) 2026-01-04 08:03:16 +08:00
fzyzcjy
c88aaf22c7 Support in-flight request age metrics for router (#16341) 2026-01-04 07:38:53 +08:00
fzyzcjy
e139d2aa76 Super tiny fix CI (#16351) 2026-01-04 07:24:46 +08:00
fzyzcjy
2337b1bbb0 Refactor and fix prefill delayer (scheduler enhancer) (#16269) 2026-01-04 07:15:06 +08:00