chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
8,056 Commits 1 Branch 0 Tags
6468cb58237942846b619409e54113c9da96d9ce
Commit Graph
188 Commits
Author SHA1 Message Date
Liangsheng Yin 933cef16cc Tiny fix mimo model conflicts with main (#15483) 2025-12-19 23:20:59 +08:00
+6
Yingchun Lai谢学扬tz李家乐张晨Shaohui Liu王晨jiangzihanxiexueyangLinghao ZhangispobockLiangsheng YinJoyFutureLiangsheng YinQiaolin Yuroot
 		160a06cab2 [Feature] Xiaomi MiMo-V2-Flash day0 support (#15207) 
Co-authored-by: 谢学扬 <xiexueyang@xiaomi.com>
Co-authored-by: tz <tangzhen3@xiaomi.com>
Co-authored-by: 李家乐 <lijiale10@xiaomi.com>
Co-authored-by: 张晨 <zhangchen50@xiaomi.com>
Co-authored-by: Shaohui Liu <liushaohui3@xiaomi.com>
Co-authored-by: 王晨 <wangchen77@xiaomi.com>
Co-authored-by: jiangzihan <jiangzihan@xiaomi.com>
Co-authored-by: xiexueyang <xyxie_wangyi@163.com>
Co-authored-by: Linghao Zhang <zhanglinghao@xiaomi.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: JoyFuture <35593546+JoyFuture@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
Co-authored-by: root <root@bj9-ml-g8h20e-k8s-slave106-20251106.alicn.idc.xiaomi.com>
 2025-12-19 11:40:07 +08:00
Charles Chen 9e0ef04e5b Support using different attention backend for draft decoding. (#14843) 2025-12-19 08:49:11 +08:00
fzyzcjy ad9616f13a Tiny extract ModelRunnerOutput (#15400) 2025-12-18 22:18:45 +08:00
Liangsheng Yin ecb401ed42 Enhance runtime memory check in CI (#15192) 2025-12-16 21:40:38 +08:00
Liwansi 30da2f0598 [NPU][eagle3] support qwen eagle3 on NPU (#14820) 2025-12-16 02:25:13 +08:00
Hanming Lu e61dabf5e4 [Qwen3-next] support mamba radix cache for overlap scheduler (#14792) 2025-12-14 18:54:16 -08:00
Liangsheng Yin ed52d01b0b Fix spec info's filter when reqs are finished right after prefill (#14742) 2025-12-14 00:32:54 +08:00
Liangsheng Yin d977dd2e07 Fix IMA with flashinfer + spec + topk & Add radix attention test cases for eagle (#13740) 2025-12-13 23:04:41 +08:00
Liangsheng Yin 7160283800 Tiny remove the duplicate function in spec v2 (#14957) 2025-12-13 11:08:27 +08:00
Rain JiangandTrevor Morris ea177372bd support mtp with deepseek r1 nvfp4 model (#13115) 
Co-authored-by: Trevor Morris <tmorris@nvidia.com>
 2025-12-06 00:45:54 -08:00
roikoren755 889b46ea50 [Spec] Mamba2 support in target models (#13434) 2025-12-06 00:50:46 +08:00
Even Zhou 894c0dc57c [NPU][1/N] NPU basic functions refactor and new modelslim quant type (#13359) 2025-12-04 16:15:31 +08:00
liupeng374andEven Zhou 2e8f54e61e [spec-overlap] bugfix for pd disaggregation and npu (#14088) 
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
 2025-12-01 22:58:20 +08:00
Liangsheng Yin 0a9d64530d Support grammar + spec + reasoning (#14163) 2025-11-30 21:19:57 +08:00
fzyzcjy b870271a50 Fix spec v2 does not support RL update weights from tensor (#14146) 2025-11-30 09:08:05 +08:00
Yixin DongandLiangsheng Yin 6350042696 feat: Naive support Spec V2 + Constrained Decoding (#13425) 
Signed-off-by: Ubospica <ubospica@gmail.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-11-27 20:31:46 +08:00
Vedant V JhaveriandLiangsheng Yin 9dab534b35 fix spec dec request level metrics (#13754) 
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-11-27 01:09:21 +08:00
Yubo Wang 18fb51583f Support FlashAttention3 page_size > 1 and topk > 1 case with paged attn and spec decode (#7725) 2025-11-26 11:44:41 +08:00
timmy-feng f33e5d1ef1 fix: spec overlap predict shape does not match verify output shapes (#12786) 2025-11-26 11:01:06 +08:00
Even Zhouandc30031083 db0ffc09ef [NPU] Fix NPU CI (#13834) 
Co-authored-by: c30031083 <chenxu140@huawei.com>
 2025-11-25 10:09:36 +08:00
yinghui 5a4394a342 align code style eagle draft&draft_extend cuda graph runner (#13533) 2025-11-23 01:55:16 +08:00
Liangsheng Yin 863124684c [Spec v2] Remove allocate_lens and enable over-allocation (#13478) 2025-11-22 22:49:10 +08:00
Even Zhou 2aec8b6e1b [Feature] Spec-Overlap supporting DP-ATTN; PD-Disaggregation; npugraph mode (#12443) 2025-11-15 21:51:07 +08:00
Zilin Zhu bc083521a8 [RL] support update_weights_from_tensor for mtp (#7415) 2025-11-15 16:57:01 +08:00
ykcombat dd192a55f4 [Feature] Enable CUDA graph for PD-Multiplexing. (#11595) 2025-11-14 03:39:40 +08:00
Zhihao Zhanganda4zhangfei 9c1c5c6d7d [ngram] use SGLANG_NGRAM_FORCE_GREEDY_VERIFY to control verify method (#13153) 
Co-authored-by: a4zhangfei <a4zhangfei@qq.com>
 2025-11-12 21:07:33 +08:00
MayDomine 3e7cc27318 At least tell the user that ngram verify is greedy! (#13039) 2025-11-12 11:29:33 +08:00
Liangsheng Yin 665416f6dd Unify memory management across (overlap, non-overlap) x (page>=1) x (spec, non-spec, spec v2) x (retract, finished) (#12224) 2025-11-11 02:56:22 +08:00
Zhihao Zhanganda4zhangfei 6f08488042 fix missing output_token_logprobs when using ngram speculative decoding (#10702) 
Co-authored-by: a4zhangfei <a4zhangfei@qq.com>
 2025-11-10 02:04:42 -08:00
Nicolas Castet 2340798353 Register allgather/reducescatter buffers with symm memory (#12572) 2025-11-04 17:11:36 -08:00
Trevor Morris dbcf85b7f0 Add --speculative-moe-runner-backend server arg (#10183) 2025-11-04 00:20:56 -08:00
Liangsheng Yin 3529c061bb [spec v2] Fix output repetition by speculative sampling error (#12561) 2025-11-03 23:00:17 +08:00
Binyao JiangandStefan He 3451fc3280 [Feature] Qwen3-Next & FLA: Support MTP topk>1; Up to 6% faster (#11133) 
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-11-01 19:47:56 -07:00
Even Zhouandliupeng374 ce6b17c0f9 [Feature] Support DeepSeek MTP on NPU (#11897) 
Co-authored-by: liupeng374 <liupeng374@huawei.com>
 2025-10-30 21:56:56 +08:00
Stefan He df08f346ce Fix Duplicate Classmethod in spec_info.py (#12354) 2025-10-29 13:57:10 -07:00
Rain H 750940ae36 Eagle3 DP attention for Qwen3 MoE (#12002) 2025-10-29 20:25:17 +08:00
Yineng Zhanggithub-actions[bot] <github-actions[bot]@users.noreply.github.com>gongwei-130
 		a6ea3add76 [Auto Sync] Update scheduler.py, spec_info.py, run_suite.py... (20251027) (#12235) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: gongwei-130 <56567052+gongwei-130@users.noreply.github.com>
 2025-10-27 17:21:08 -07:00
Liangsheng Yin 97710ccd1a Fix flush cache API for spec v2 (#11918) 2025-10-21 23:01:16 +08:00
Cheng Wan bfc3b3f786 [9/N] MoE Refactor: cleanup dispatcher interfaces (#11847) 2025-10-20 10:11:46 -07:00
PaiiiiandPaiiiiiiiiiiiiii efa473348b [Spec Decoding] Support MTP for dsv3.2 (#11652) 
Co-authored-by: Paiiiiiiiiiiiiii <zengpai@baidu.com>
 2025-10-19 23:44:22 +08:00
Marin 1dacedd2db make sure logit bias is applied during eagle spec decoding verification (#11555) 2025-10-19 10:53:33 +08:00
Liangsheng Yin a93f10a722 [overlap-spec] support page size > 1 (#11772) 2025-10-19 02:09:13 +08:00
Zilin Zhu 69fe3c9726 Manually flip deepep_mode for cuda_graph (#11666) 2025-10-18 08:05:48 +08:00
Chang Su 627974405d [Lint] Add python/sglang to ruff F401 checks and remove unused imports in files (#11685) 2025-10-17 16:49:46 -07:00
Liangsheng Yin cde5a6e30f Abstraction for spec worker and code cleanup (#11643) 2025-10-17 23:31:36 +08:00
Liangsheng Yin d88ac9bc9a [overlap-spec] Make plan stream an option (#11724) 2025-10-17 15:48:57 +08:00
Lianmin Zheng cd7e1bd591 Sync code and test CI; rename some env vars (#11686) 2025-10-15 18:37:03 -07:00
Shangming CaiandLiangsheng Yin 6d0364681c Fix 1-step draft model forward (#11653) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-10-15 19:11:33 +08:00
Liangsheng Yin 8221f9ae8b Tiny cleanup some eagle unused codes (#11660) 2025-10-15 17:24:08 +08:00