Commit Graph
8979 Commits
Author SHA1 Message Date
Mick 09491a9bcd cli: support sglang version (#17250) 2026-01-18 13:20:24 +08:00
Alison Shao 7edb06158e Add runner utilization report workflow (#17234) 2026-01-17 19:28:05 -08:00
Lancer e486a4dac1 [diffusion] feat: support default 4-step inference for Flux2-Klein distilled models (#17225)
Signed-off-by: Lancer <maruixiang6688@gmail.com>
2026-01-18 10:15:50 +08:00
Hudson Xing 90399cbc07 fix(ci): recover from corrupted MMMU parquet cache (#17256) 2026-01-17 17:32:01 -08:00
Michael 53609e5e5b Revert "[Diffusion] Move diffusion time embedding to jit kernel" (#17257) 2026-01-17 21:29:22 +08:00
fzyzcjy 9c2530642c Change routing policy API to be async to support more policies (#17048) 2026-01-17 17:12:51 +08:00
Zehuan Li d2c863878c [DLLM] Implement initial dynamic batching for diffusion LLM (#14883) 2026-01-17 16:48:15 +08:00
Yi ZhangandLiangsheng Yin 737a1183d6 [BUGFIX] fix radix cache memory consumption to avoid OOM (#17191)
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
2026-01-17 16:47:37 +08:00
Mick dc743fe4ba [diffusion] chore: clean srt imports (#17252) 2026-01-17 15:47:49 +08:00
Nan Jiang dd99f818e0 fix: fix regression and unclear pattern (#16561) 2026-01-16 23:21:42 -08:00
Hudson Xing 8ce64aa155 fix(ci): skip offline mode for LoRA scenarios (#17248) 2026-01-16 23:06:20 -08:00
Mick eb768189ee [diffusion] doc: add instruction for adding performance baseline of new model (#17249) 2026-01-17 12:24:59 +08:00
Xiaoyu Zhang 2cdd4370bc [Diffusion] Move diffusion time embedding to jit kernel (#16879) 2026-01-17 12:21:22 +08:00
fzyzcjy a7b5f75d88 Support integration tests with Redis binary (#17045) 2026-01-17 11:59:04 +08:00
fzyzcjy 305c1a575a Tiny remove unused code (#17047) 2026-01-17 11:58:43 +08:00
fzyzcjy c824ddd581 Fix imbalancedness for manual policy min group mode for requests without routing id (#17044) 2026-01-17 11:58:35 +08:00
siyu e18e0057a4 Add an env var to allow transferring small metadata via TCP for PD (#16951) 2026-01-16 18:32:22 -08:00
zijiexia 166396ca4c [Docs] minor update on ep docs (#17242) 2026-01-16 18:20:47 -08:00
Stefan He 43779f27b7 Disable PCG for draft worker (#16354) 2026-01-17 09:56:56 +08:00
Baizhou Zhang 8b9e9357fe [2/n] deepseek_v2.py Refactor: Migrate MHA forward method in deepseek_v2.py (#16817) 2026-01-17 09:36:25 +08:00
b8zhong d36f6f043c [Fix] flashinfer_trtllm intermediate_size assertion with Qwen3 + TP=8 (#16824) 2026-01-17 09:24:05 +08:00
Mohammad Miadh Angkad b0701f02b3 Fix benchmark import for should_use_tensor_core (#17232) 2026-01-16 17:48:36 -05:00
Chang Su 4229de3b13 [model-gateway] Refine TokenizerRegisty.load() to handle duplication (#17230) 2026-01-16 13:53:21 -08:00
Alison Shao 2e14407983 Increase 5090 test parallelism from 4 to 8 (#17233) 2026-01-16 13:52:56 -08:00
Douglas Yang d2ec128bbf fix: ci failure monitor reorganization (#17165) 2026-01-16 13:25:13 -08:00
Lingjun WenandXinyuan Tong 7f8353aff3 [BugFix]: Fix sglang.bench_one_batch (#16925)
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-01-16 13:15:34 -08:00
Chang Su a7f5677abe refactor: unify registration through tokenizer_registration workflow (#17187) 2026-01-16 10:53:36 -08:00
Makcum888e 3e968ab369 [Refactor] [CI] Remove redundant CI test runs (#17217) 2026-01-16 09:52:06 -08:00
Alison Shao b4fce9955a Add CI Coverage Overview workflow with detailed test listings (#16842) 2026-01-16 09:42:50 -08:00
Yi Zhong ec9b48ea96 Add olmo3 in supported docs (#13672)
Signed-off-by: Vincent Zhong <207368749+vincentzed@users.noreply.github.com>
2026-01-16 12:18:16 -05:00
Baizhou Zhang a04675892e Update flashinfer to 0.6.1 (#15551) 2026-01-17 00:48:30 +08:00
Yongfei Xu 82a1b645ba [DeepSeek V3.1/V3.2] Optimize fused moe configs for H20 & H20-3E based on swapab (#17133) 2026-01-17 00:10:52 +08:00
billishyahao 6f10e17b4a [bugfix] fix qwen3-next alt_stream none issue (#17016) 2026-01-16 22:40:25 +08:00
Mohammad Miadh Angkad c771933dc5 [Doc] Tiny docs update for CUDA 13 (#17200) 2026-01-16 20:53:36 +08:00
Nicolas Castet 9d8bbd4223 Add clear error message when OOM with symmetric memory (#17038) 2026-01-16 20:44:37 +08:00
Raghav Ravishankar daea51385d Add AFMoE model implementation (#13216) 2026-01-16 20:35:42 +08:00
3355b6e21b feat: add request queued timeout (#17143)
Co-authored-by: qiuxuan.lzw <qiuxuan.lzw@alibaba-inc.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
2026-01-16 17:55:09 +08:00
R0CKSTAR a1dd3d48ac [diffusion] hardware: support diffusion (single GPU, 3/N) (#17105)
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
2026-01-16 17:01:09 +08:00
Zhiqiang Xie d9ed80b9f1 fix AMD CI failure of NUMA binding (#17184) 2026-01-16 00:21:04 -08:00
Adarsh Shirawalmath 7c39ea68f3 [diffusion] model: support flux Klein (#17173) 2026-01-16 16:16:17 +08:00
YAMY daa4841e86 [ConfigArgumentMerger] Improve ConfigArgumentMerger compatibility with external callers (#17051) 2026-01-16 15:32:42 +08:00
YC Tseng 968c4f55b1 [AMD] Enable DeepseekV3.2 test for AMD CI (#16934) 2026-01-15 21:58:46 -08:00
Chang Su 669d309a8b [model-gateway] Consolidate "unknown" model id usage (#17186) 2026-01-15 21:02:40 -08:00
Hudson Xing 21ee597e4a ci: enable offline mode when local cache is complete to avoid HF Hub … (#16121) 2026-01-15 20:15:33 -08:00
Xiaoyu Zhang 6ee970a365 [Diffusion] Hot fix broken output_path default value (#17180) 2026-01-16 12:14:09 +08:00
shuwenn 8ec160ed46 feature: support uvicorn access log filter(disable logging /metrics) (#15513) 2026-01-15 20:00:06 -08:00
YAMY 2740ed1ae7 [eval] GSM8k support for run_eval (#17041) 2026-01-16 11:10:17 +08:00
b8zhong d44f09ad98 [Benchmark] Add GSM8K Platinum Eval (#14565) 2026-01-16 11:06:14 +08:00
Lianmin ZhengandLiangsheng Yin e7dc85c50b Fix grammar sync across TP ranks (#17100)
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
2026-01-15 18:38:01 -08:00
Ratish P c81bad1bf7 [diffusion] feat: add cloud storage support for API (#14579) 2026-01-16 10:29:38 +08:00