chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
9,135 Commits 1 Branch 0 Tags
7ace64d1d88de1dfd3cfb6622f3fcbba4fe7eef8
Commit Graph

204 Commits

Author SHA1 Message Date
Ke Bao
ce8a6ac690 Evict swa kv cache during decoding (#17220) 2026-01-19 22:36:52 +08:00
YAMY
a45e0e5df4 [SPEC_V2] Enable cudagraph draft_extend for trtllm_mla_backend and Acclen Fix for DP under cudagraph mode (#16974) 2026-01-18 15:56:21 +08:00
Ke Bao
7f8a58fffb Refactor prefix cache type checking (#17028) 2026-01-15 11:28:13 +08:00
chenxu214
53dca74f47 Bugfix: EagleDraftWorker has not attribute "eagle_use_aux_hidden_state" (#16480) 2026-01-12 20:14:35 +08:00
chenxu140
38a88479c6 llama model and llama eagle3 model support dp-attn (#15268) 2026-01-12 08:54:56 +08:00
Liangsheng Yin
75da784d48 Tiny simplify draft worker init. (#16446) 2026-01-09 11:20:29 +08:00
Lianmin Zheng
fb04e7e3c8 [Auto Sync] Update schedule_batch.py, common.py, eagle_info... (20260105) (#16519) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: cctry <shiyang@x.ai>
 2026-01-08 02:35:46 -08:00
YAMY
12cde0df99 [SPEC_V2] Fix Acclen drop when enabling DP Attention for Spec-Overlap (#16310) 2026-01-05 11:39:23 +08:00
Izzy Putterman
3dfff6ae3c Eagle: GPT-OSS Eagle v2 support (#14920) 
Signed-off-by: Izzy Putterman <iputterman@nvidia.com>
 2025-12-30 14:23:07 -08:00
Liangsheng Yin
ba67e006a7 Refactor speculative algorithm registry. (#16168) 2025-12-31 01:24:22 +08:00
Gaoji Liu
7518dc3532 feat(SpecEagleV2): add standalone_worker_v2 (#12625) 
Co-authored-by: liugaoji.lgj <liugaoji.lgj@alibaba-inc.com>
 2025-12-30 17:55:04 +08:00
Liangsheng Yin
be3828a13b Tiny cleanup duplicate code for multi-layer eagle worker. (#16004) 2025-12-28 18:08:20 +08:00
Ke Bao
7b7e357f61 Separate swa and local attention chunk cache eviction (#15820) 2025-12-26 09:34:22 +08:00
Liangsheng Yin
cb719c74ad [2/N] clean duplicate code of logprob processing in spec. (#15593) 2025-12-24 15:21:13 +08:00
Liangsheng Yin
3c882db3ad Adjust wrong mtp meaning introduce by mimo (#15632) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-23 02:06:46 +08:00
Liangsheng Yin
8766a1dd24 [1 / N] Clean up logprob utils (#15509) 2025-12-22 03:12:25 +08:00
Liangsheng Yin
933cef16cc Tiny fix mimo model conflicts with main (#15483) 2025-12-19 23:20:59 +08:00
Yingchun Lai
160a06cab2 [Feature] Xiaomi MiMo-V2-Flash day0 support (#15207) 
Co-authored-by: 谢学扬 <xiexueyang@xiaomi.com>
Co-authored-by: tz <tangzhen3@xiaomi.com>
Co-authored-by: 李家乐 <lijiale10@xiaomi.com>
Co-authored-by: 张晨 <zhangchen50@xiaomi.com>
Co-authored-by: Shaohui Liu <liushaohui3@xiaomi.com>
Co-authored-by: 王晨 <wangchen77@xiaomi.com>
Co-authored-by: jiangzihan <jiangzihan@xiaomi.com>
Co-authored-by: xiexueyang <xyxie_wangyi@163.com>
Co-authored-by: Linghao Zhang <zhanglinghao@xiaomi.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: JoyFuture <35593546+JoyFuture@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
Co-authored-by: root <root@bj9-ml-g8h20e-k8s-slave106-20251106.alicn.idc.xiaomi.com>
 2025-12-19 11:40:07 +08:00
Charles Chen
9e0ef04e5b Support using different attention backend for draft decoding. (#14843) 2025-12-19 08:49:11 +08:00
fzyzcjy
ad9616f13a Tiny extract ModelRunnerOutput (#15400) 2025-12-18 22:18:45 +08:00
Liangsheng Yin
ecb401ed42 Enhance runtime memory check in CI (#15192) 2025-12-16 21:40:38 +08:00
Liwansi
30da2f0598 [NPU][eagle3] support qwen eagle3 on NPU (#14820) 2025-12-16 02:25:13 +08:00
Hanming Lu
e61dabf5e4 [Qwen3-next] support mamba radix cache for overlap scheduler (#14792) 2025-12-14 18:54:16 -08:00
Liangsheng Yin
ed52d01b0b Fix spec info's filter when reqs are finished right after prefill (#14742) 2025-12-14 00:32:54 +08:00
Liangsheng Yin
d977dd2e07 Fix IMA with flashinfer + spec + topk & Add radix attention test cases for eagle (#13740) 2025-12-13 23:04:41 +08:00
Liangsheng Yin
7160283800 Tiny remove the duplicate function in spec v2 (#14957) 2025-12-13 11:08:27 +08:00
Rain Jiang
ea177372bd support mtp with deepseek r1 nvfp4 model (#13115) 
Co-authored-by: Trevor Morris <tmorris@nvidia.com>
 2025-12-06 00:45:54 -08:00
roikoren755
889b46ea50 [Spec] Mamba2 support in target models (#13434) 2025-12-06 00:50:46 +08:00
Even Zhou
894c0dc57c [NPU][1/N] NPU basic functions refactor and new modelslim quant type (#13359) 2025-12-04 16:15:31 +08:00
liupeng374
2e8f54e61e [spec-overlap] bugfix for pd disaggregation and npu (#14088) 
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
 2025-12-01 22:58:20 +08:00
Liangsheng Yin
0a9d64530d Support grammar + spec + reasoning (#14163) 2025-11-30 21:19:57 +08:00
fzyzcjy
b870271a50 Fix spec v2 does not support RL update weights from tensor (#14146) 2025-11-30 09:08:05 +08:00
Yixin Dong
6350042696 feat: Naive support Spec V2 + Constrained Decoding (#13425) 
Signed-off-by: Ubospica <ubospica@gmail.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-11-27 20:31:46 +08:00
Vedant V Jhaveri
9dab534b35 fix spec dec request level metrics (#13754) 
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-11-27 01:09:21 +08:00
Yubo Wang
18fb51583f Support FlashAttention3 page_size > 1 and topk > 1 case with paged attn and spec decode (#7725) 2025-11-26 11:44:41 +08:00
timmy-feng
f33e5d1ef1 fix: spec overlap predict shape does not match verify output shapes (#12786) 2025-11-26 11:01:06 +08:00
Even Zhou
db0ffc09ef [NPU] Fix NPU CI (#13834) 
Co-authored-by: c30031083 <chenxu140@huawei.com>
 2025-11-25 10:09:36 +08:00
yinghui
5a4394a342 align code style eagle draft&draft_extend cuda graph runner (#13533) 2025-11-23 01:55:16 +08:00
Liangsheng Yin
863124684c [Spec v2] Remove allocate_lens and enable over-allocation (#13478) 2025-11-22 22:49:10 +08:00
Even Zhou
2aec8b6e1b [Feature] Spec-Overlap supporting DP-ATTN; PD-Disaggregation; npugraph mode (#12443) 2025-11-15 21:51:07 +08:00
Zilin Zhu
bc083521a8 [RL] support update_weights_from_tensor for mtp (#7415) 2025-11-15 16:57:01 +08:00
ykcombat
dd192a55f4 [Feature] Enable CUDA graph for PD-Multiplexing. (#11595) 2025-11-14 03:39:40 +08:00
Zhihao Zhang
9c1c5c6d7d [ngram] use SGLANG_NGRAM_FORCE_GREEDY_VERIFY to control verify method (#13153) 
Co-authored-by: a4zhangfei <a4zhangfei@qq.com>
 2025-11-12 21:07:33 +08:00
MayDomine
3e7cc27318 At least tell the user that ngram verify is greedy! (#13039) 2025-11-12 11:29:33 +08:00
Liangsheng Yin
665416f6dd Unify memory management across (overlap, non-overlap) x (page>=1) x (spec, non-spec, spec v2) x (retract, finished) (#12224) 2025-11-11 02:56:22 +08:00
Zhihao Zhang
6f08488042 fix missing output_token_logprobs when using ngram speculative decoding (#10702) 
Co-authored-by: a4zhangfei <a4zhangfei@qq.com>
 2025-11-10 02:04:42 -08:00
Nicolas Castet
2340798353 Register allgather/reducescatter buffers with symm memory (#12572) 2025-11-04 17:11:36 -08:00
Trevor Morris
dbcf85b7f0 Add --speculative-moe-runner-backend server arg (#10183) 2025-11-04 00:20:56 -08:00
Liangsheng Yin
3529c061bb [spec v2] Fix output repetition by speculative sampling error (#12561) 2025-11-03 23:00:17 +08:00
Binyao Jiang
3451fc3280 [Feature] Qwen3-Next & FLA: Support MTP topk>1; Up to 6% faster (#11133) 
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-11-01 19:47:56 -07:00