chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
8,008 Commits 1 Branch 0 Tags
f6c9db4bc433fcab1060c235a9eb742ba3f2dc66
Commit Graph
1375 Commits
Author SHA1 Message Date
Zehuan Li f6c9db4bc4 [DLLM] Fix dLLM regression (#15371) 2025-12-19 10:55:46 +08:00
Zehuan Li b2803ff207 [DLLM] Add CI for diffusion LLMs (#14723) 2025-12-19 08:54:03 +08:00
Feng Su 29e8f7f9e5 multimodal: precompute hash for MultimodalDataItem (#14354) 
Signed-off-by: Feng Su <sufeng@linux.alibaba.com>
Signed-off-by: Junjie Mao <junjie.mao@linux.alibaba.com>
 2025-12-18 15:27:59 -08:00
fzyzcjy 88a405cc10 Support EPLB balancedness prometheus metric without GPU->CPU synchronize (#15401) 2025-12-18 22:24:23 +08:00
fzyzcjy 602fe3b296 Super tiny add moe_ep_rank to prometheus labels (#15407) 2025-12-18 22:23:20 +08:00
fzyzcjy ad9616f13a Tiny extract ModelRunnerOutput (#15400) 2025-12-18 22:18:45 +08:00
fzyzcjy c5f4e20f2f Support GPU execution time breakdown by forward mode metrics (#15396) 2025-12-18 22:18:17 +08:00
Lianmin Zheng d1f0063262 Clean up __init__ function of the scheduler and event loop for PD (#15298) 2025-12-18 01:35:14 -08:00
Xuchun ShangandShangming Cai fea2d5211d [bug fix][pp] fix inconsistent latency between tp (#15379) 
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-12-18 16:59:56 +08:00
Shangming Cai ee1ca51de8 [PP] Fix dynamic chunking strategy for PP (#15372) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-12-18 14:24:55 +08:00
Scott Lee 169a75dfb8 Add request-level timestamp for when prefill finishes (#14860) 2025-12-17 13:34:46 -08:00
Liangsheng Yin 0c00220795 tiny unify environ usage (#15335) 2025-12-17 23:31:43 +08:00
Shangming Cai eeb2b9b259 [PP] Minor code cleanup for Pipeline Parallelism (#15329) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-12-17 23:31:25 +08:00
fzyzcjy 3e690cce53 Add realtime token counter metrics (#15198) 2025-12-17 21:20:24 +08:00
fzyzcjy b12b40dea4 Add cuda_graph_forward_passes_total and num_retracted_reqs_total (#15189) 2025-12-17 21:19:42 +08:00
Xuchun Shangkun-llflKun
 		45a959d3e9 [PP] Add pp support for Qwen3-VL (#12333) 
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
Signed-off-by: Kun(llfl) <i@imux.top>
Signed-off-by: Kun(llfl) <llfl@linux.alibaba.com>
Co-authored-by: kun-llfl <i@imux.top>
Co-authored-by: Kun(llfl) <llfl@linux.alibaba.com>
 2025-12-17 16:03:58 +08:00
luqitao 0129c911e0 fix(function_call): fallback to decode when batch decode options differ (#15155) 
Signed-off-by: luqitao <luqt@awcloud.com>
 2025-12-16 20:21:08 -08:00
Lianmin Zheng 9d64a7b24f Minor style fixes to the scheduler.py (#15218) 2025-12-16 17:09:44 -08:00
amysaq2023 ccc8f3b266 support non disturbing remote instance weight loader v2 (#14997) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-16 14:39:56 -08:00
Liangsheng Yin ecb401ed42 Enhance runtime memory check in CI (#15192) 2025-12-16 21:40:38 +08:00
Yuan Luoandluoyuan.luo 3912ee4991 [VLM] feat: support chunked vit attention (#14907) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-15 12:11:02 +08:00
Hanming Lu e61dabf5e4 [Qwen3-next] support mamba radix cache for overlap scheduler (#14792) 2025-12-14 18:54:16 -08:00
Tianyu Guoliusy58ZhengWGNicholasShangming CaiYuhao Yang
 		9acb21ae27 feat: support EPD disaggregation (#12263) 
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
Co-authored-by: ZhengWG <zwg0606@gmail.com>
Co-authored-by: Nicholas <45984215+liusy58@users.noreply.github.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
 2025-12-14 22:30:08 +08:00
liupeng374 f50af32d8b [scheduler] remove scheduler allgather for best throughout (#14294) 2025-12-14 16:37:49 +08:00
fzyzcjy 168a31eb00 Support prefill max requests limitation (#14993) 2025-12-14 10:52:10 +08:00
fzyzcjy fdc93b019c Add sglang:decode_sum_seq_lens metric (#15066) 2025-12-14 08:42:45 +08:00
liupeng374 3134d2b2a7 [scheduler] enhance scheduler in dp_attention mixed case with spec (#14201) 2025-12-14 02:52:26 +08:00
Liangsheng Yin ed52d01b0b Fix spec info's filter when reqs are finished right after prefill (#14742) 2025-12-14 00:32:54 +08:00
Liangsheng Yin 01e3b3f3a3 Fix decode OOM caused by retraction (#14939) 2025-12-13 12:59:17 +08:00
fzyzcjy 313f59ad80 Add soft watchdogs to debug soft hangs (#15023) 2025-12-13 10:41:35 +08:00
fzyzcjy 487cf81a68 Tiny extract SchedulerWatchdog (#15021) 2025-12-13 10:40:32 +08:00
fzyzcjy 8cc77261ec Super tiny remove unused argument (#14966) 2025-12-13 09:41:01 +08:00
Chenxi Li 9b9d21312a Feature/Fix multi lora scheduler blocking issue and evict LoRA None lastly (#14795) 2025-12-12 17:13:05 -08:00
Yineng Zhangandfzyzcjy 4b7b5af36a Revert several PRs (#14958) 
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-12-12 11:25:12 -08:00
Kevin LiandShangming Cai 8fa8d9d7e8 [PD] Add decode PP event loop for PD disaggregation (#14945) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-12-12 20:15:00 +08:00
Xuchun ShangLianmin ZhengShangming Caibluecoffee8zhangxiaolei123456ybyang
 		c01b2ee094 [PP] Refactor PP to async mode (#11852) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: bluecoffee8 <jasperli2002@gmail.com>
Co-authored-by: zhangxiaolei123456 <zhangxiaolei.666@bytedance.com>
Co-authored-by: ybyang <10629930+whybeyoung@users.noreply.github.com>
 2025-12-12 12:54:16 +08:00
Xinyuan Tong 9975acf50f [refactor] Update reasoning parameter to require_reasoning (#14922) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 16:55:59 -08:00
amysaq2023 70758d457e support non-disturbing remote-instance-weight-loader (#13125) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-11 16:45:32 -08:00
Liangsheng Yin bd7824b24d Minor code style fix for dllm (#14836) 2025-12-11 10:35:42 +09:00
Lianmin Zhengandgithub-actions[bot] <github-actions[bot]@users.noreply.github.com> 4285e99da7 [Auto Sync] Update data_parallel_controller.py, detokenizer... (20251209) (#14759) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-12-09 18:38:38 -08:00
ShawnY112358and赵晨阳 f077436831 [fix] Fix issues for in-flight weight updates (#14064) 
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
 2025-12-09 18:12:19 -08:00
Liangsheng Yinandzhanghaotong cbc7dcdaa7 Re-add the API serving timing metrics. (#14744) 
Signed-off-by: zhanghaotong <zhanghaotong.zht@antgroup.com>
Co-authored-by: zhanghaotong <zhanghaotong.zht@antgroup.com>
 2025-12-10 10:17:48 +09:00
Lianmin Zheng 036e64dafa move multi-item scoring functions in tokenizer manager into a separate file (#14740) 2025-12-09 14:47:06 -08:00
b8zhong 55504df2f7 Add FP8 Blockwise GEMM Backend Flag --fp8-gemm-backend (#14379) 2025-12-09 12:05:56 -08:00
Lianmin Zheng ab0048793c Revert "[Feat] Add received_time in serving_base" (#14743) 2025-12-09 06:52:27 -08:00
zhanghaotong fe7f91ef82 [Feat] Add received_time in serving_base (#13432) 
Signed-off-by: zhanghaotong <zhanghaotong.zht@antgroup.com>
 2025-12-09 22:00:24 +09:00
kun-llfl cef5ba65b1 [Bugfix] Fix environ error in scheduler_runtime_checker_mixin.py (#14461) 
Signed-off-by: Kun(llfl) <i@imux.top>
 2025-12-09 20:33:41 +08:00
Lianmin ZhengandByron Hsu ce4e836be5 Add per-request decode tp size (#14678) 
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
 2025-12-08 20:24:31 -08:00
Yuhao Yang b9bef31a15 fix: use .get() when accessing strict mem-check env variable (#14657) 2025-12-08 13:25:42 -08:00
Muqi LiandXinyuan Tong 06836ad02a [Reasoning + Structured Output] make reasoning compatible with structured output (#12551) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-08 01:28:39 -08:00