chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
8,883 Commits 1 Branch 0 Tags
ba625c2d908ab10427bb9b4d459ad8902d127007
Commit Graph

502 Commits

Author SHA1 Message Date
shaharmor98
ba625c2d90 Feat/support nemotron h mtp (#17013) 
Signed-off-by: Shahar Mor <smor@nvidia.com>
 2026-01-14 16:30:35 +08:00
Gaoji Liu
7b682de870 [Model] Support IQuest-Coder-40B-Loop (#16348) 
Co-authored-by: yxing <yxing@iquestlab.com>
Co-authored-by: yzhu <yzhu@ubiquant.com>
Co-authored-by: zelong518 <zelonghuang02@gmail.com>
 2026-01-12 23:44:45 +08:00
Yibo Cai
2f4a6addf3 [cpu/arm64] support run sglang on arm64 cpu (#14867) 2026-01-11 04:27:19 -08:00
Yuwei An
2a9344d320 [tiny remove] remove torch_compile in parallel_state (#16865) 
Signed-off-by: Oasis-Git <ayw.sirius19@gmail.com>
 2026-01-10 16:04:25 -08:00
YAMY
2d088b85d9 [IDLE FORWARD][Indexer] Fix forward_idle bs mismatch issue in DeepseekV3.2's NSAIndexer (#15227) 2026-01-10 18:14:30 +08:00
lg(x)
3a8b44fe89 Update LoRA Weights via Tensor (#16226) 
Co-authored-by: PopSoda2002 <zhouhp.me@gmail.com>
 2026-01-10 17:36:43 +08:00
Liangsheng Yin
e6d40bff81 Revert "feat: reduce constrained-decoding overhead in TP" (#16845) 2026-01-10 11:39:38 +08:00
Raayan Dhar
76b3c698d6 feat: reduce constrained-decoding overhead in TP (#13947) 
Signed-off-by: Raayan Dhar raayan.dhar@gmail.com <raayan.dhar@gmail.com>
Signed-off-by: raayandhar <raayan.dhar@gmail.com>
 2026-01-10 00:38:32 +08:00
Junrong Lin
eec7dbd31e remove redundant max_running_reqs calculation in r3 (#16629) 2026-01-08 09:45:21 +08:00
Izzy Putterman
3dfff6ae3c Eagle: GPT-OSS Eagle v2 support (#14920) 
Signed-off-by: Izzy Putterman <iputterman@nvidia.com>
 2025-12-30 14:23:07 -08:00
Ke Bao
059428bd8a Tiny remove additional args in init_memory_pool (#16158) 2025-12-30 21:38:06 +08:00
Ke Bao
3de23274ee Clean up swa handling in fa3 backend (#15877) 2025-12-30 08:51:40 +08:00
Liangsheng Yin
cb1812954a Introduce ModelRunnerKVCacheMixin to simplify the code. (#15821) 2025-12-26 11:17:33 +08:00
Yingchun Lai
bb9e6cdf9e [MiMoV2Flash] fix: respect --swa-full-tokens-ratio arg (#15488) 2025-12-25 21:02:56 +08:00
Liangsheng Yin
f4e835af2f Cleanup ModelRunner (#15802) 2025-12-25 18:13:30 +08:00
Liangsheng Yin
d5fbbfd988 Retry removing wrong logic about max total token in spec decoding (#15748) 2025-12-24 21:13:26 +08:00
Baizhou Zhang
468931b572 [Tiny]Move deepseek fp4 cutlass moe test to per-commit test (#15565) 2025-12-21 18:08:07 -08:00
Junrong Lin
bed301a5ac [Feature] Enable return routed experts (#12162) 
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-12-21 15:16:43 +08:00
Yuwei An
9d0347b33a EP Support for Piecewise Cuda Graph (#14164) 
Signed-off-by: Oasis-Git <ayw.sirius19@gmail.com>
 2025-12-20 01:59:27 +08:00
Mayyyy
2c5a446006 [NPU]Fix for ipc handle with npu (#14138) 2025-12-19 22:39:04 +08:00
Qiaolin Yu
173940927f Add customized sampler registration (#15423) 2025-12-18 23:10:23 -08:00
Yingchun Lai
160a06cab2 [Feature] Xiaomi MiMo-V2-Flash day0 support (#15207) 
Co-authored-by: 谢学扬 <xiexueyang@xiaomi.com>
Co-authored-by: tz <tangzhen3@xiaomi.com>
Co-authored-by: 李家乐 <lijiale10@xiaomi.com>
Co-authored-by: 张晨 <zhangchen50@xiaomi.com>
Co-authored-by: Shaohui Liu <liushaohui3@xiaomi.com>
Co-authored-by: 王晨 <wangchen77@xiaomi.com>
Co-authored-by: jiangzihan <jiangzihan@xiaomi.com>
Co-authored-by: xiexueyang <xyxie_wangyi@163.com>
Co-authored-by: Linghao Zhang <zhanglinghao@xiaomi.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: JoyFuture <35593546+JoyFuture@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
Co-authored-by: root <root@bj9-ml-g8h20e-k8s-slave106-20251106.alicn.idc.xiaomi.com>
 2025-12-19 11:40:07 +08:00
Charles Chen
9e0ef04e5b Support using different attention backend for draft decoding. (#14843) 2025-12-19 08:49:11 +08:00
fzyzcjy
88a405cc10 Support EPLB balancedness prometheus metric without GPU->CPU synchronize (#15401) 2025-12-18 22:24:23 +08:00
fzyzcjy
ad9616f13a Tiny extract ModelRunnerOutput (#15400) 2025-12-18 22:18:45 +08:00
Cheng Wan
5290cef97c Fix condition check for require_gathered_buffer (#15328) 2025-12-17 12:42:47 -08:00
Liangsheng Yin
0c00220795 tiny unify environ usage (#15335) 2025-12-17 23:31:43 +08:00
elvischenv
feb8e30b9d [Hotfix] Fix required enable_mamba_track argument for Flashinfer autotune path (#15314) 2025-12-17 17:03:24 +08:00
elvischenv
435d1c83c1 [Perf] Enable Flashinfer autotune by default (#14357) 2025-12-16 23:01:39 -08:00
amysaq2023
ccc8f3b266 support non disturbing remote instance weight loader v2 (#14997) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-16 14:39:56 -08:00
Shangming Cai
36fcf71fff [Qwen3-next] Add PD disaggregation support for mamba with extra_buffer (#15180) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: ybyang <10629930+whybeyoung@users.noreply.github.com>
 2025-12-16 14:36:00 +08:00
Hanming Lu
e61dabf5e4 [Qwen3-next] support mamba radix cache for overlap scheduler (#14792) 2025-12-14 18:54:16 -08:00
Chunyuan WU
2a39cfe0ff call check_quantized_moe_compatibility after initialize (#13876) 2025-12-13 13:32:19 +08:00
Liangsheng Yin
01e3b3f3a3 Fix decode OOM caused by retraction (#14939) 2025-12-13 12:59:17 +08:00
Yineng Zhang
4b7b5af36a Revert several PRs (#14958) 
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-12-12 11:25:12 -08:00
Liangsheng Yin
c660d8dfd0 Re-org eagle unit tests (#14909) 2025-12-12 12:25:39 +09:00
amysaq2023
70758d457e support non-disturbing remote-instance-weight-loader (#13125) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-11 16:45:32 -08:00
Yinghai Lu
b05b346a13 [loader] enable private loader (#14620) 2025-12-11 11:49:46 -08:00
yuchengz816-bot
e54307f26a [6/n] Fix num_token_non_padded computation in prefill (#14313) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Runkai Tao <rt572@physics.rutger.edu>
 2025-12-10 19:15:19 -08:00
Peng Zhang
21028b5507 [RL] support weight reload for low-bit rollout (#9650) 
Co-authored-by: Hecate0821 <hec4te0821@gmail.com>
Co-authored-by: eternally-z <zzywzj@gmail.com>
Co-authored-by: Wilboludriver <wilbolu@outlook.com>
Co-authored-by: Wilbolu <81792854+Wilboludriver@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
 2025-12-10 15:44:01 +08:00
kk
c106b54b57 Aiter fp8 kv cache (#13147) 2025-12-08 16:39:53 -08:00
roikoren755
2ce121a1c3 Enable RadixCache for Mamba2 models (#13584) 2025-12-05 18:23:58 +08:00
Xinyuan Tong
6d37e70883 ministral3 (#14251) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Yueming Yuan <yy28@illinois.edu>
 2025-12-04 14:31:26 -08:00
Even Zhou
894c0dc57c [NPU][1/N] NPU basic functions refactor and new modelslim quant type (#13359) 2025-12-04 16:15:31 +08:00
Lianmin Zheng
64092c8b55 [Auto Sync] Rename is_hybrid to is_hybrid_swa (#14252) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: Hanming Lu <hanming@x.ai>
 2025-12-01 23:24:24 -08:00
Stefan He
8fe8b63576 Revert "Try to remove wrong logic about max total token in spec decoding" (#14259) 2025-12-01 18:18:03 -08:00
Yuan Luo
26aebf83d3 [VLM] Support Piecewise CUDA Graph for Qwen3-Omni-MOE (#14222) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-02 10:12:10 +08:00
fzyzcjy
f4a0c5c76b Try to remove wrong logic about max total token in spec decoding (#14167) 2025-12-01 15:29:58 +08:00
Minglei Zhu
e8542db558 [piecewise] move piecewise_cuda_graph_runner init to model_runner initialize (#14034) 
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
Co-authored-by: Binyao Jiang <byjiang1996@gmail.com>
 2025-11-30 22:16:04 -08:00
Byron Hsu
0825d7f4c6 [piecewise] Refactor VLM to support input embed buffer and remove external embedder hack (#14155) 2025-11-30 21:43:09 -08:00