chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
4,763 Commits 1 Branch 0 Tags
c9bf3877a0a02a80267bd851fb712c30f3bf9ccd
Commit Graph

626 Commits

Author SHA1 Message Date
Even Zhou
de2dd73831 Revert "[feature] Rework Ascend NPU graph support" (#9385) 2025-08-20 00:35:10 -07:00
Even Zhou
3680d6f88b [feature] Rework Ascend NPU graph support (#9350) 
Co-authored-by: ronnie_zheng <zl19940307@163.com>
Co-authored-by: yezhifeng (D) <y00897525@china.huawei.com>
Co-authored-by: anon189Ty <Stari_Falcon@outlook.com>
Co-authored-by: Maksim <makcum888e@mail.ru>
Co-authored-by: ssshinigami <44640852+ssshinigami@users.noreply.github.com>
 2025-08-19 20:32:27 -07:00
Even Zhou
f4fafacc5d Revert "[feature] Ascend NPU graph support (#8027)" (#9348) 2025-08-19 10:11:23 -07:00
zxy
a31ea44824 support for interns1-mini (#9299) 2025-08-18 17:56:04 -07:00
Binyao Jiang
c2fbf60f39 [GLM4.1V and GLM4.5V] Add vision transformer num_dummy_head support: max tp=4 -> max tp=8 (#9059) 2025-08-18 14:40:13 -07:00
blzheng
ebbb75e917 [CPU] Fix TP padding issue on Phi-4 (#8289) 2025-08-17 16:25:26 -07:00
Netanel Haber
3d77a31885 from python.sglang.srt -> from sglang.srt (#9268) 2025-08-17 02:45:45 -07:00
Netanel Haber
845d12a979 model: support nvidia/Llama-3_3-Nemotron-Super-49B-v1 (#9067) 
Co-authored-by: Kyle Huang <kylhuang@nvidia.com>
 2025-08-17 01:48:15 -07:00
Stefan He
e47800e176 Quick Fix GLM (#9264) 2025-08-16 23:43:41 -07:00
Shangming Cai
384f8ab5ce [PD] Support PD disaggregation with Prefill PP (#8846) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: root <huzhiyuan@xiaohongshu.com>
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
Co-authored-by: Francis <38564764+ssssnow@users.noreply.github.com>
Co-authored-by: zitto <zhjc1124@gmail.com>
 2025-08-16 18:31:31 -07:00
VDV1985
94371dbbd6 [feature] Ascend NPU graph support (#8027) 
Co-authored-by: ronnie_zheng <zl19940307@163.com>
Co-authored-by: yezhifeng (D) <y00897525@china.huawei.com>
Co-authored-by: anon189Ty <Stari_Falcon@outlook.com>
Co-authored-by: Maksim <makcum888e@mail.ru>
Co-authored-by: ssshinigami <44640852+ssshinigami@users.noreply.github.com>
 2025-08-16 17:25:17 -07:00
Trevor Morris
eff4eb3fdd Add fp4 quantize before all-gather for Flashinfer cutlass MoE DP (max throughput) (#7667) 2025-08-15 22:08:11 -07:00
Cheng Wan
84b006b278 Cleanup MoE Refactor (#9223) 2025-08-15 02:28:33 -07:00
Cheng Wan
295895120d [6/N] MoE Refactor: Cleanup MoE-related configs (#8849) 2025-08-14 21:14:53 -07:00
wxzhoucs
4c22897a66 Feature: support qwen and llama4 reducescatter for dp attention padding (#9101) 2025-08-13 21:10:29 -07:00
Cheng Wan
b87aacb5c5 [DP Attention] Refactor: adding some utility functions (#9136) 2025-08-13 21:08:06 -07:00
Nicolas Castet
6b7c24712c Fix broken trtllm_mha attn backend with gpt-oss (#9161) 2025-08-13 16:40:55 -07:00
Zhihao Liu
65736dc524 [Model] Support Qwen3ForSequenceClassification for Qwen3-Embed Model (#7957) 2025-08-13 11:14:54 -07:00
Ke Bao
0ff6d1fce1 Support FA3 backend for gpt-oss (#9028) 2025-08-13 10:41:50 -07:00
Kevin Xiang Li
3b3b3baf9f Double vision prefill throughput by defaulting to optimal vision attention backend (#8484) 
Co-authored-by: Xiang (Kevin) Li <lik@nvidia.com>
 2025-08-13 02:08:30 -07:00
fzyzcjy
9394ed6386 Fix gpt-oss ~2x memory consumption issue (#9146) 2025-08-13 00:11:43 -07:00
Jiaqi Gu
c9ee738515 Fuse writing KV buffer into rope kernel (part 2: srt) (#9014) 
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
 2025-08-12 13:15:30 -07:00
Faraz
f508cd3cb7 TRTLLM-MLA FP8 path (#8638) 
Signed-off-by: Faraz Khoubsirat <58580514+farazkh80@users.noreply.github.com>
 2025-08-11 14:02:13 -07:00
Xiaoyu Zhang
44e86480e8 fuse allreduce and residual_rmsnorm (#8731) 2025-08-11 13:50:53 -07:00
Lifu Huang
f8a173bb50 Improve LoRA Perf by Deprecating FlashInfer and Eliminating Redundant Tensor Ops (#8940) 2025-08-10 01:04:45 -07:00
Lianmin Zheng
9a44b643c6 Fix CI (#9012) 2025-08-09 13:33:42 -07:00
Brayden Zhong
4a9f3eef90 Tiny Llama4 type error in constructor (#6752) 2025-08-09 01:03:59 -07:00
Binyao Jiang
f29aba8c6e Support glm4.1v and glm4.5v (#8798) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
Co-authored-by: Chang Su <csu272@usc.edu>
 2025-08-09 00:59:13 -07:00
Binyao Jiang
7b81f956eb Fix qwen2 audio not working bug (#8600) 2025-08-09 00:42:29 -07:00
fzyzcjy
d3e67deb1b Fix redundant kernel in sink dtype conversion (#8966) 2025-08-09 00:34:49 -07:00
Lifu Huang
6e2151183b Fix incorrect default get_hidden_dim logic (#8987) 2025-08-09 00:25:38 -07:00
Wenbo Yang
1132547496 Add ernie4.py for ERNIE-4.5 (#7657) 2025-08-08 00:55:48 -07:00
Cheng Wan
1d24db8348 Expert Parallelism for GPT-OSS (#8944) 2025-08-08 00:46:42 -07:00
Stefan He
d3be97104b correct the tp_plan logic (#8850) 2025-08-07 16:53:34 -07:00
Xiaoyu Zhang
47824c1488 [Perf] Auto enable best flashinfer mxfp4 kernel in b200 (#8898) 2025-08-07 01:08:41 -07:00
PGFLMG
b7cd743038 [Feat] QWen-1M context support[2/2]: Update block sparse attention backend (#5949) 2025-08-06 23:49:36 -07:00
eigen
6ad6c8c9e6 feat: openai oss attention sink support with trtllm-gen backend #8825 (#8834) 
Co-authored-by: averyhuang <averyh@nvidia.com>
 2025-08-06 19:18:27 -07:00
Cheng Wan
5b6acc1495 fix glm4 moe (#8883) 2025-08-06 18:02:31 -07:00
Trevor Morris
c0e84297c2 Use reduce scatter for DP (#8539) 2025-08-06 16:21:26 -07:00
Ke Bao
399e7ec8b3 Refine naming (#8868) 2025-08-06 21:37:02 +08:00
Ying Sheng
168033d5fb Support mxfp4 for GPT-OSS (#8843) 
Co-authored-by: Co-author fzyzcjy <ch271828n@outlook.com>
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
Co-authored-by: zhuofan1123 <zhuofanl@nvidia.com>
Co-authored-by: liz-badada <jinyanc@nvidia.com>
Co-authored-by: xutizhou <xutingz@nvidia.com>
Co-authored-by: linhu-nv <linhu@nvidia.com>
 2025-08-06 00:05:25 -07:00
Stefan He
89588179cf [1/3] Optimize Slime Update Weights: Remove QWen3MOE Load Weight Overhead (#8751) 2025-08-05 22:07:54 -07:00
Praneth Paruchuri
d26ca84f39 Support bailing moe (#8680) 2025-08-05 20:40:34 -07:00
Ying Sheng
c1d2061f97 Add initial support for gpt-oss (#8824) 2025-08-05 13:42:01 -07:00
Yineng Zhang
901ab758ec chore: upgrade transformers 4.55.0 (#8823) 
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
 2025-08-05 11:37:21 -07:00
Yuxuan Zhang
a4b0d5c9e5 GLM-4.5 and GLM-4.5-Air both support (#8804) 2025-08-05 03:29:20 -07:00
kk
d4bf5a8524 Support OCP MXFP4 quantization on AMD GPUs (#8255) 
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Hubert Lu <Hubert.Lu@amd.com>
 2025-08-04 18:14:52 -07:00
Lifu Huang
7cb20754fa [Fix] Fix several issues preventing gemma3n LoRA support. (#8776) 2025-08-04 17:11:46 -07:00
azhurkevich
915140fd18 [NVIDIA] Add Low Latency NVFP4 decode kernels from Flashinfer (#8552) 
Co-authored-by: Cheng Wan <cwan@x.ai>
 2025-08-04 03:10:02 -07:00
Yuxuan Zhang
760286e3d3 use fp32 for e_score_correction_bias in GLM-4.5 (#8729) 2025-08-03 10:43:40 -07:00