chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
8,134 Commits 1 Branch 0 Tags
fa2966983a04584f381a4d144c97a9e595eb0ac1
Commit Graph

529 Commits

Author SHA1 Message Date
Tianyu Guo
fa2966983a Support PP for zmq_to_scheduler (#15312) 2025-12-23 17:07:55 +08:00
Liangsheng Yin
3c882db3ad Adjust wrong mtp meaning introduce by mimo (#15632) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-23 02:06:46 +08:00
fzyzcjy
454a2544f2 Support soft watchdog for tokenizer/detokenizer/dp-controller processes (#15607) 2025-12-22 16:59:16 +08:00
Junrong Lin
bed301a5ac [Feature] Enable return routed experts (#12162) 
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-12-21 15:16:43 +08:00
Liangsheng Yin
933cef16cc Tiny fix mimo model conflicts with main (#15483) 2025-12-19 23:20:59 +08:00
Lianmin Zheng
f228b662a7 Update readme (#15425) 2025-12-18 23:06:00 -08:00
Yingchun Lai
160a06cab2 [Feature] Xiaomi MiMo-V2-Flash day0 support (#15207) 
Co-authored-by: 谢学扬 <xiexueyang@xiaomi.com>
Co-authored-by: tz <tangzhen3@xiaomi.com>
Co-authored-by: 李家乐 <lijiale10@xiaomi.com>
Co-authored-by: 张晨 <zhangchen50@xiaomi.com>
Co-authored-by: Shaohui Liu <liushaohui3@xiaomi.com>
Co-authored-by: 王晨 <wangchen77@xiaomi.com>
Co-authored-by: jiangzihan <jiangzihan@xiaomi.com>
Co-authored-by: xiexueyang <xyxie_wangyi@163.com>
Co-authored-by: Linghao Zhang <zhanglinghao@xiaomi.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: JoyFuture <35593546+JoyFuture@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
Co-authored-by: root <root@bj9-ml-g8h20e-k8s-slave106-20251106.alicn.idc.xiaomi.com>
 2025-12-19 11:40:07 +08:00
Zehuan Li
f6c9db4bc4 [DLLM] Fix dLLM regression (#15371) 2025-12-19 10:55:46 +08:00
fzyzcjy
88a405cc10 Support EPLB balancedness prometheus metric without GPU->CPU synchronize (#15401) 2025-12-18 22:24:23 +08:00
fzyzcjy
c5f4e20f2f Support GPU execution time breakdown by forward mode metrics (#15396) 2025-12-18 22:18:17 +08:00
Lianmin Zheng
d1f0063262 Clean up __init__ function of the scheduler and event loop for PD (#15298) 2025-12-18 01:35:14 -08:00
fzyzcjy
b12b40dea4 Add cuda_graph_forward_passes_total and num_retracted_reqs_total (#15189) 2025-12-17 21:19:42 +08:00
Lianmin Zheng
9d64a7b24f Minor style fixes to the scheduler.py (#15218) 2025-12-16 17:09:44 -08:00
amysaq2023
ccc8f3b266 support non disturbing remote instance weight loader v2 (#14997) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-16 14:39:56 -08:00
Hanming Lu
e61dabf5e4 [Qwen3-next] support mamba radix cache for overlap scheduler (#14792) 2025-12-14 18:54:16 -08:00
Tianyu Guo
9acb21ae27 feat: support EPD disaggregation (#12263) 
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
Co-authored-by: ZhengWG <zwg0606@gmail.com>
Co-authored-by: Nicholas <45984215+liusy58@users.noreply.github.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
 2025-12-14 22:30:08 +08:00
fzyzcjy
168a31eb00 Support prefill max requests limitation (#14993) 2025-12-14 10:52:10 +08:00
liupeng374
3134d2b2a7 [scheduler] enhance scheduler in dp_attention mixed case with spec (#14201) 2025-12-14 02:52:26 +08:00
Liangsheng Yin
ed52d01b0b Fix spec info's filter when reqs are finished right after prefill (#14742) 2025-12-14 00:32:54 +08:00
Liangsheng Yin
01e3b3f3a3 Fix decode OOM caused by retraction (#14939) 2025-12-13 12:59:17 +08:00
fzyzcjy
313f59ad80 Add soft watchdogs to debug soft hangs (#15023) 2025-12-13 10:41:35 +08:00
fzyzcjy
487cf81a68 Tiny extract SchedulerWatchdog (#15021) 2025-12-13 10:40:32 +08:00
Chenxi Li
9b9d21312a Feature/Fix multi lora scheduler blocking issue and evict LoRA None lastly (#14795) 2025-12-12 17:13:05 -08:00
Yineng Zhang
4b7b5af36a Revert several PRs (#14958) 
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-12-12 11:25:12 -08:00
Kevin Li
8fa8d9d7e8 [PD] Add decode PP event loop for PD disaggregation (#14945) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-12-12 20:15:00 +08:00
Xuchun Shang
c01b2ee094 [PP] Refactor PP to async mode (#11852) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: bluecoffee8 <jasperli2002@gmail.com>
Co-authored-by: zhangxiaolei123456 <zhangxiaolei.666@bytedance.com>
Co-authored-by: ybyang <10629930+whybeyoung@users.noreply.github.com>
 2025-12-12 12:54:16 +08:00
Xinyuan Tong
9975acf50f [refactor] Update reasoning parameter to require_reasoning (#14922) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 16:55:59 -08:00
amysaq2023
70758d457e support non-disturbing remote-instance-weight-loader (#13125) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-11 16:45:32 -08:00
ShawnY112358
f077436831 [fix] Fix issues for in-flight weight updates (#14064) 
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
 2025-12-09 18:12:19 -08:00
b8zhong
55504df2f7 Add FP8 Blockwise GEMM Backend Flag --fp8-gemm-backend (#14379) 2025-12-09 12:05:56 -08:00
Muqi Li
06836ad02a [Reasoning + Structured Output] make reasoning compatible with structured output (#12551) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-08 01:28:39 -08:00
roikoren755
2ce121a1c3 Enable RadixCache for Mamba2 models (#13584) 2025-12-05 18:23:58 +08:00
Lianmin Zheng
64092c8b55 [Auto Sync] Rename is_hybrid to is_hybrid_swa (#14252) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: Hanming Lu <hanming@x.ai>
 2025-12-01 23:24:24 -08:00
liupeng374
2e8f54e61e [spec-overlap] bugfix for pd disaggregation and npu (#14088) 
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
 2025-12-01 22:58:20 +08:00
fzyzcjy
340c613ab5 Support numactl bind for CPU and memory before process starts (#14156) 2025-11-30 17:00:33 +08:00
fzyzcjy
45cf575852 Fix overlap scheduler not take effect when outputing logprobs (#14096) 2025-11-28 18:15:56 +08:00
vipwangerxiao
ab9a46d462 Support configuring the request limit per receiving poll (#14076) 
Co-authored-by: Peng Wang <peng_wang@linux.alibaba.com>
Co-authored-by: Feng Su <225349073+sufeng-buaa@users.noreply.github.com>
 2025-11-28 16:14:21 +08:00
Yixin Dong
6350042696 feat: Naive support Spec V2 + Constrained Decoding (#13425) 
Signed-off-by: Ubospica <ubospica@gmail.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-11-27 20:31:46 +08:00
fzyzcjy
25758647b1 Support sanity checking weight consistency especially for RL (#13854) 2025-11-27 20:25:12 +08:00
Liangsheng Yin
eff6a07c8f Fix get_load API (#13991) 2025-11-26 22:07:51 +08:00
vipwangerxiao
15729dbc8e Use dynamically maintained num_waiting_tokens in get_load() (#13203) 
Signed-off-by: Peng Wang <peng_wang@linux.alibaba.com>
Co-authored-by: Peng Wang <peng_wang@linux.alibaba.com>
 2025-11-26 20:34:45 +08:00
Zehuan Li
21b0582d4b [feature] Initial block diffusion language model support (#12588) 
Co-authored-by: Tiwei Bie <tiwei.btw@antgroup.com>
 2025-11-26 17:57:54 +08:00
ShawnY112358
007c3e234c [feat] support in-flight weight update (#10071) 
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
 2025-11-25 22:03:13 -08:00
ant-yy
a146f833f1 [Fix]: Adjust FutureMap's token_id_bufs Size to Prevent ChunkedPrefill's next_token_ids from Overwriting Previous Prefill Requests' next_token_id (#13713) 
Signed-off-by: vito.yy <vito.yy@antgroup.com>
 2025-11-25 01:08:52 +08:00
Liangsheng Yin
b2f7b08c49 Refactor cache init logic (#13800) 2025-11-24 11:41:46 +08:00
Liangsheng Yin
5c2915494c [Scheduler] Tiny organize code style (#13806) 2025-11-23 23:41:34 +08:00
Liangsheng Yin
863124684c [Spec v2] Remove allocate_lens and enable over-allocation (#13478) 2025-11-22 22:49:10 +08:00
Liangsheng Yin
3e804bb0ba Update mem checker during busy (#13704) 2025-11-22 21:15:02 +08:00
Michele Marzollo
b30f63c40f [Bugfix] Fix hidden state size in EAGLE PD disaggregation buffers (#13590) 
Co-authored-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-11-21 15:32:40 +08:00
Even Zhou
2aec8b6e1b [Feature] Spec-Overlap supporting DP-ATTN; PD-Disaggregation; npugraph mode (#12443) 2025-11-15 21:51:07 +08:00