chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
7,927 Commits 1 Branch 0 Tags
ccc8f3b266faa278f7ef531d75727c2b37e1a364
Commit Graph

682 Commits

Author SHA1 Message Date
amysaq2023
ccc8f3b266 support non disturbing remote instance weight loader v2 (#14997) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-16 14:39:56 -08:00
Shangming Cai
36fcf71fff [Qwen3-next] Add PD disaggregation support for mamba with extra_buffer (#15180) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: ybyang <10629930+whybeyoung@users.noreply.github.com>
 2025-12-16 14:36:00 +08:00
Hanming Lu
e61dabf5e4 [Qwen3-next] support mamba radix cache for overlap scheduler (#14792) 2025-12-14 18:54:16 -08:00
narutolhy
69cfb17b9a [Fix] avoid stream sync in _compute_mrope_positions (#14956) 2025-12-13 18:50:57 -08:00
liupeng374
d36299ad77 [NPU] perf update with kvcache nz & w4a8 quant (#14423) 2025-12-13 17:39:55 +08:00
Chunyuan WU
2a39cfe0ff call check_quantized_moe_compatibility after initialize (#13876) 2025-12-13 13:32:19 +08:00
Liangsheng Yin
01e3b3f3a3 Fix decode OOM caused by retraction (#14939) 2025-12-13 12:59:17 +08:00
Yineng Zhang
4b7b5af36a Revert several PRs (#14958) 
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-12-12 11:25:12 -08:00
Liangsheng Yin
c660d8dfd0 Re-org eagle unit tests (#14909) 2025-12-12 12:25:39 +09:00
Yuhao Yao
e9e7f15eb5 [bugfix] fix TBO crashes when attn_tp_size > 1 (#13730) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-12-11 18:18:40 -08:00
amysaq2023
70758d457e support non-disturbing remote-instance-weight-loader (#13125) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-11 16:45:32 -08:00
Yinghai Lu
b05b346a13 [loader] enable private loader (#14620) 2025-12-11 11:49:46 -08:00
Vladimir221
27032cecd9 [Ascend]Support of piecewise graph compilation for prefill on NPU (#12287) 
Co-authored-by: ronnie_zheng <zl19940307@163.com>
 2025-12-11 21:10:07 +08:00
yuchengz816-bot
e54307f26a [6/n] Fix num_token_non_padded computation in prefill (#14313) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Runkai Tao <rt572@physics.rutger.edu>
 2025-12-10 19:15:19 -08:00
Liangsheng Yin
bd7824b24d Minor code style fix for dllm (#14836) 2025-12-11 10:35:42 +09:00
Peng Zhang
21028b5507 [RL] support weight reload for low-bit rollout (#9650) 
Co-authored-by: Hecate0821 <hec4te0821@gmail.com>
Co-authored-by: eternally-z <zzywzj@gmail.com>
Co-authored-by: Wilboludriver <wilbolu@outlook.com>
Co-authored-by: Wilbolu <81792854+Wilboludriver@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
 2025-12-10 15:44:01 +08:00
kk
c106b54b57 Aiter fp8 kv cache (#13147) 2025-12-08 16:39:53 -08:00
Tiwei Bie
36361adcbf [DLLM] Add initial cuda graph support (#14203) 2025-12-08 14:12:35 +08:00
Liangsheng Yin
125e17efd5 Add small model test for spec v2 + dp + trtllm_mla (#14576) 2025-12-07 23:55:00 +08:00
Rain Jiang
ea177372bd support mtp with deepseek r1 nvfp4 model (#13115) 
Co-authored-by: Trevor Morris <tmorris@nvidia.com>
 2025-12-06 00:45:54 -08:00
roikoren755
2ce121a1c3 Enable RadixCache for Mamba2 models (#13584) 2025-12-05 18:23:58 +08:00
Minglei Zhu
b76e303e6a clean up gemlite usage (#14444) 2025-12-04 21:52:56 -08:00
Xinyuan Tong
6d37e70883 ministral3 (#14251) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Yueming Yuan <yy28@illinois.edu>
 2025-12-04 14:31:26 -08:00
Even Zhou
894c0dc57c [NPU][1/N] NPU basic functions refactor and new modelslim quant type (#13359) 2025-12-04 16:15:31 +08:00
Lianmin Zheng
ca52ed425f Clean up imports and move files (#14317) 2025-12-02 16:31:54 -08:00
Quanfeng Li
427b08e24d Init TBO with dp_padded batch (#11423) 
Co-authored-by: Cheng Wan <wan4ch@gmail.com>
Co-authored-by: Yuhao Yao <37280700+yuhyao@users.noreply.github.com>
 2025-12-02 10:34:26 -08:00
alisonshao
0141ca370f Revert PR #14044: Restore separate memory pool for piecewise CUDA graph (#14278) 2025-12-02 09:53:16 -08:00
Lianmin Zheng
64092c8b55 [Auto Sync] Rename is_hybrid to is_hybrid_swa (#14252) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: Hanming Lu <hanming@x.ai>
 2025-12-01 23:24:24 -08:00
Stefan He
8fe8b63576 Revert "Try to remove wrong logic about max total token in spec decoding" (#14259) 2025-12-01 18:18:03 -08:00
Yuan Luo
26aebf83d3 [VLM] Support Piecewise CUDA Graph for Qwen3-Omni-MOE (#14222) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-02 10:12:10 +08:00
fzyzcjy
f4a0c5c76b Try to remove wrong logic about max total token in spec decoding (#14167) 2025-12-01 15:29:58 +08:00
Binyao Jiang
0f8e53947d [Piecewise] Use same global graph memory pool as the main cuda graph … (#14044) 
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
Co-authored-by: BBuf <1182563586@qq.com>
 2025-11-30 23:04:10 -08:00
Minglei Zhu
e8542db558 [piecewise] move piecewise_cuda_graph_runner init to model_runner initialize (#14034) 
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
Co-authored-by: Binyao Jiang <byjiang1996@gmail.com>
 2025-11-30 22:16:04 -08:00
Byron Hsu
0825d7f4c6 [piecewise] Refactor VLM to support input embed buffer and remove external embedder hack (#14155) 2025-11-30 21:43:09 -08:00
elvischenv
848ee57067 feat: support flashinfer kernel autotune (#12306) 
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
 2025-11-29 00:05:37 -08:00
fzyzcjy
25758647b1 Support sanity checking weight consistency especially for RL (#13854) 2025-11-27 20:25:12 +08:00
Stefan He
9f340ab1fb [Piecewise] support disable decode cuda graph when enable piecewise cuda graph (#13965) 2025-11-26 18:35:59 -08:00
ShawnY112358
5155016b56 [feat] update bucketed weights from distributed (#13824) 
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-11-26 15:30:45 -08:00
Netanel Haber
082b54c689 Support nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-BF16 (and nvidia/C-RADIOv2-H) (#12277) 2025-11-26 16:28:52 -07:00
Liangsheng Yin
6c190cbda0 Rename: --hooks to --forward-hooks (#13994) 2025-11-26 22:26:28 +08:00
Zehuan Li
21b0582d4b [feature] Initial block diffusion language model support (#12588) 
Co-authored-by: Tiwei Bie <tiwei.btw@antgroup.com>
 2025-11-26 17:57:54 +08:00
StonyPort
540d6fee20 Support piecewise CUDA graph for embedding models (#13852) 
Co-authored-by: qiuxuan.lzw <qiuxuan.lzw@alibaba-inc.com>
 2025-11-26 15:29:46 +08:00
Lianmin Zheng
1ab6ce0e62 [Auto Sync] Improve profilers and simplify bench_one_batch_server.py (#13866) 2025-11-25 12:13:31 -08:00
Chen1022
d64bf6c6ce Support piecewise cuda graph for Qwen3-next (#13081) 2025-11-25 21:01:27 +08:00
Liwansi
432ecf841e [Ascend] qwen optimization (#12078) 
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
 2025-11-25 19:44:24 +08:00
gongwei-130
8ff3ef1fef fix: draft model revision misuse model revision (#11893) 2025-11-24 21:13:37 -08:00
Even Zhou
db0ffc09ef [NPU] Fix NPU CI (#13834) 
Co-authored-by: c30031083 <chenxu140@huawei.com>
 2025-11-25 10:09:36 +08:00
fzyzcjy
94216a9cc4 Fix quantized moe checker fail for Qwen3 dense fp8 model (#13853) 2025-11-24 11:16:50 -08:00
YAMY
98b38de3f2 Fix: Safe RoPE Cache Expansion to Prevent Position-ID Out-of-Bounds in EAGLE + Long-Sequence Workloads (#11871) 2025-11-25 01:19:06 +08:00
Yuan Luo
8ef11569a2 [VLM] Revise InternVL Piecewise CUDA Graph Supporting (#13846) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-24 22:15:10 +08:00