chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
7,725 Commits 1 Branch 0 Tags
9975acf50f220208e2e813c73f322f24af285130
Commit Graph

711 Commits

Author SHA1 Message Date
amysaq2023
70758d457e support non-disturbing remote-instance-weight-loader (#13125) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-11 16:45:32 -08:00
Yinghai Lu
b05b346a13 [loader] enable private loader (#14620) 2025-12-11 11:49:46 -08:00
Vladimir221
27032cecd9 [Ascend]Support of piecewise graph compilation for prefill on NPU (#12287) 
Co-authored-by: ronnie_zheng <zl19940307@163.com>
 2025-12-11 21:10:07 +08:00
b8zhong
da9b801eb7 fix lora target all + csgmv backend (#14796) 2025-12-10 11:42:51 -08:00
Peng Zhang
21028b5507 [RL] support weight reload for low-bit rollout (#9650) 
Co-authored-by: Hecate0821 <hec4te0821@gmail.com>
Co-authored-by: eternally-z <zzywzj@gmail.com>
Co-authored-by: Wilboludriver <wilbolu@outlook.com>
Co-authored-by: Wilbolu <81792854+Wilboludriver@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
 2025-12-10 15:44:01 +08:00
TomerBN-Nvidia
b1cbfce612 fix server args bug (#14725) 2025-12-09 21:05:05 -08:00
Lianmin Zheng
18bd8e8d6d Improve CI by trying a warmup before unit tests (#14669) 2025-12-09 15:17:59 -08:00
b8zhong
55504df2f7 Add FP8 Blockwise GEMM Backend Flag --fp8-gemm-backend (#14379) 2025-12-09 12:05:56 -08:00
Francis
07404d7689 [HiCache] fix condition check when use decode offload (#14489) 2025-12-08 15:52:40 -08:00
b8zhong
32f8b6064e improve default glm mtp setting (#14457) 
Signed-off-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-08 13:27:13 -08:00
Tiwei Bie
36361adcbf [DLLM] Add initial cuda graph support (#14203) 2025-12-08 14:12:35 +08:00
Vladimir Serov
f124539a01 [NPU]LoRA: Adding Torch Native backend (#14132) 2025-12-08 02:16:07 +08:00
Yuan Luo
26d95008b6 [apply][2/2] Fused qk_norm_rope for Qwen3-MoE (#13998) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-07 20:25:18 +08:00
Tiwei Bie
9abcab3ffa [DLLM] feat: Add threshold based parallel decoding support (#14412) 
Co-authored-by: Jinwei Yao <jinweiy@illinois.edu>
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
 2025-12-07 18:25:33 +08:00
Chen1022
3c7886ec4c Fix attention backend logic for Qwen3-Next on SM100 (#14560) 2025-12-06 22:03:34 -08:00
Rain Jiang
ea177372bd support mtp with deepseek r1 nvfp4 model (#13115) 
Co-authored-by: Trevor Morris <tmorris@nvidia.com>
 2025-12-06 00:45:54 -08:00
Baizhou Zhang
42fcf5438f Revert "tiny remove deprecated endpoint call" (#14533) 2025-12-05 23:48:54 -08:00
fzyzcjy
3d1b591aa1 Tiny use trtllm_mha as default when possible (#14291) 2025-12-05 14:26:03 -08:00
b8zhong
ec7b2c16d9 tiny remove deprecated endpoint call (#13607) 2025-12-05 09:54:49 -08:00
elvischenv
205f041e96 Add Mistral Large 3 Eagle Support (#14466) 
Co-authored-by: Linda-Stadter <57756729+Linda-Stadter@users.noreply.github.com>
 2025-12-05 23:11:41 +08:00
roikoren755
2ce121a1c3 Enable RadixCache for Mamba2 models (#13584) 2025-12-05 18:23:58 +08:00
Sam
922756aaa1 [FIX] trtllm-moe-fp4-renorm for Qwen series models (#14350) 2025-12-04 12:52:21 -08:00
Daniel Cámpora
8428078436 Add Mistral Large 3 support. (#14213) 
Co-authored-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Linda-Stadter <57756729+Linda-Stadter@users.noreply.github.com>
 2025-12-04 20:00:05 +08:00
Even Zhou
894c0dc57c [NPU][1/N] NPU basic functions refactor and new modelslim quant type (#13359) 2025-12-04 16:15:31 +08:00
TomerBN-Nvidia
02af51e4fc Support fp4 fp8 non gated moe (#13794) 
Co-authored-by: Roi Koren <roik@nvidia.com>
Co-authored-by: Tomer Natan <tbarnatan@computelab-frontend-8.nvidia.com>
 2025-12-01 15:26:28 -08:00
YAMY
decb48965d [DeepSeekV3.2] Enable pure TP & Partial DP Attention (#13646) 2025-11-30 15:59:23 -08:00
Yuhao Yang
f03ea34a3d add runtime check for PyTorch 2.9.1 + CuDNN < 9.15 to prevent Conv3d performance issues (#14119) 2025-11-29 10:05:54 -05:00
elvischenv
848ee57067 feat: support flashinfer kernel autotune (#12306) 
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
 2025-11-29 00:05:37 -08:00
Cheng Wan
0fe74af563 Remove incorrect deep_gemm assertions from server_args.py (#14113) 
Co-authored-by: Kangyan Zhou <zky314343421@gmail.com>
 2025-11-28 20:25:39 -08:00
fjybiocs
143b57b805 enable piecewise cuda graph for prefill server (#13377) 
Co-authored-by: serverance.fu <serverance.fu@temu.com>
 2025-11-29 12:09:26 +08:00
Baizhou Zhang
7ab548ef64 [2/2] Refactor DeepGeem requant for FP8 FusedMoE on Blackwell (#13960) 2025-11-27 09:00:26 -05:00
Sam
91e8dc371a [Feat][NVFP4] Enable NVFP4 MoE for Qwen series models (eg. Qwen3-Next) #13761 (#13761) 
Co-authored-by: Kaixi Hou <kaixih@nvidia.com>
 2025-11-26 17:53:45 -07:00
Lianmin Zheng
231df4b0d4 Cleanup server args (#14027) 2025-11-26 16:32:41 -08:00
Liangsheng Yin
6c190cbda0 Rename: --hooks to --forward-hooks (#13994) 2025-11-26 22:26:28 +08:00
Zehuan Li
21b0582d4b [feature] Initial block diffusion language model support (#12588) 
Co-authored-by: Tiwei Bie <tiwei.btw@antgroup.com>
 2025-11-26 17:57:54 +08:00
Yubo Wang
18fb51583f Support FlashAttention3 page_size > 1 and topk > 1 case with paged attn and spec decode (#7725) 2025-11-26 11:44:41 +08:00
Jan Bernlöhr
fcccaf9001 Add Llama4 attention backend auto-selection (#13421) 
Signed-off-by: jbernloehr <jbernloehr@nvidia.com>
 2025-11-25 11:54:21 -08:00
Liwansi
432ecf841e [Ascend] qwen optimization (#12078) 
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
 2025-11-25 19:44:24 +08:00
gongwei-130
8ff3ef1fef fix: draft model revision misuse model revision (#11893) 2025-11-24 21:13:37 -08:00
Nicolas Castet
b0a26ba624 Add support for bf16 x bf16 cutlass fused MoE (#10275) 
Co-authored-by: Sam Li <lsam@nvidia.com>
Co-authored-by: jackeyhua <jackeyhuasjtu@gmail.com>
 2025-11-24 18:49:39 -08:00
hlu1
618ca23802 [Deepseek] Refactor deepseek server_args _handle_model_specific_adjustments (#13687) 2025-11-23 12:41:14 -08:00
Binyao Jiang
b29769f3b6 Move unnecessary input_addr capture under debug mode flag for speed-up (#13690) 2025-11-22 11:42:26 -08:00
Chen Haozhe
6c2e5fcd91 [feat][Ascend][Mindspore]: support model-impl of mindspore (#9234) 2025-11-19 09:17:47 +08:00
b8zhong
92ad2ff9ce Flashinfer TRTLLM-GEN-MoE + Qwen3 (#13489) 2025-11-18 14:18:29 -08:00
Nicholas
ac81db66c2 [VLM][feat] Support encoder DP for Qwen2.5-VL (#13126) 
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: liusy58 <xiehang.lsy@alibaba-inc.com>
Co-authored-by: Yuan Luo <yuan.luo@hotmail.com>
 2025-11-18 16:13:18 +08:00
wingedge
f1be8aa0f2 chore: add an unified server arg for multimodal inputs preprocess config(#12149) 
Co-authored-by: bianfeng <bianfeng@pinduoduo.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-11-18 12:18:50 +08:00
Carlo Mussolini
df56139226 Adding user defined hooks support (#13217) 2025-11-17 23:07:37 +08:00
kebyn
15db5497d3 [feature] Custom base path on FastAPI server (#5879) 
Co-authored-by: lianhu.yin <lianhu.yin@nio.com>
Co-authored-by: kebyn <kebyn@kebyn.cc>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-11-17 22:40:09 +08:00
huangtingwei
1dcde53928 [HiCache] support memory_pool_host page head layout (#11644) 2025-11-17 13:45:17 +08:00
lixiaolx
d368c7451a (1/n)support context parallel with deepseekv3.2-DSA (#12065) 2025-11-16 20:12:25 -08:00