chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
4,535 Commits 1 Branch 0 Tags
9a44b643c67d10bcb4d8129e2153647a26558ae5
Commit Graph

601 Commits

Author SHA1 Message Date
Lianmin Zheng
9a44b643c6 Fix CI (#9012) 2025-08-09 13:33:42 -07:00
Brayden Zhong
4a9f3eef90 Tiny Llama4 type error in constructor (#6752) 2025-08-09 01:03:59 -07:00
Binyao Jiang
f29aba8c6e Support glm4.1v and glm4.5v (#8798) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
Co-authored-by: Chang Su <csu272@usc.edu>
 2025-08-09 00:59:13 -07:00
Binyao Jiang
7b81f956eb Fix qwen2 audio not working bug (#8600) 2025-08-09 00:42:29 -07:00
fzyzcjy
d3e67deb1b Fix redundant kernel in sink dtype conversion (#8966) 2025-08-09 00:34:49 -07:00
Lifu Huang
6e2151183b Fix incorrect default get_hidden_dim logic (#8987) 2025-08-09 00:25:38 -07:00
Wenbo Yang
1132547496 Add ernie4.py for ERNIE-4.5 (#7657) 2025-08-08 00:55:48 -07:00
Cheng Wan
1d24db8348 Expert Parallelism for GPT-OSS (#8944) 2025-08-08 00:46:42 -07:00
Stefan He
d3be97104b correct the tp_plan logic (#8850) 2025-08-07 16:53:34 -07:00
Xiaoyu Zhang
47824c1488 [Perf] Auto enable best flashinfer mxfp4 kernel in b200 (#8898) 2025-08-07 01:08:41 -07:00
PGFLMG
b7cd743038 [Feat] QWen-1M context support[2/2]: Update block sparse attention backend (#5949) 2025-08-06 23:49:36 -07:00
eigen
6ad6c8c9e6 feat: openai oss attention sink support with trtllm-gen backend #8825 (#8834) 
Co-authored-by: averyhuang <averyh@nvidia.com>
 2025-08-06 19:18:27 -07:00
Cheng Wan
5b6acc1495 fix glm4 moe (#8883) 2025-08-06 18:02:31 -07:00
Trevor Morris
c0e84297c2 Use reduce scatter for DP (#8539) 2025-08-06 16:21:26 -07:00
Ke Bao
399e7ec8b3 Refine naming (#8868) 2025-08-06 21:37:02 +08:00
Ying Sheng
168033d5fb Support mxfp4 for GPT-OSS (#8843) 
Co-authored-by: Co-author fzyzcjy <ch271828n@outlook.com>
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
Co-authored-by: zhuofan1123 <zhuofanl@nvidia.com>
Co-authored-by: liz-badada <jinyanc@nvidia.com>
Co-authored-by: xutizhou <xutingz@nvidia.com>
Co-authored-by: linhu-nv <linhu@nvidia.com>
 2025-08-06 00:05:25 -07:00
Stefan He
89588179cf [1/3] Optimize Slime Update Weights: Remove QWen3MOE Load Weight Overhead (#8751) 2025-08-05 22:07:54 -07:00
Praneth Paruchuri
d26ca84f39 Support bailing moe (#8680) 2025-08-05 20:40:34 -07:00
Ying Sheng
c1d2061f97 Add initial support for gpt-oss (#8824) 2025-08-05 13:42:01 -07:00
Yineng Zhang
901ab758ec chore: upgrade transformers 4.55.0 (#8823) 
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
 2025-08-05 11:37:21 -07:00
Yuxuan Zhang
a4b0d5c9e5 GLM-4.5 and GLM-4.5-Air both support (#8804) 2025-08-05 03:29:20 -07:00
kk
d4bf5a8524 Support OCP MXFP4 quantization on AMD GPUs (#8255) 
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Hubert Lu <Hubert.Lu@amd.com>
 2025-08-04 18:14:52 -07:00
Lifu Huang
7cb20754fa [Fix] Fix several issues preventing gemma3n LoRA support. (#8776) 2025-08-04 17:11:46 -07:00
azhurkevich
915140fd18 [NVIDIA] Add Low Latency NVFP4 decode kernels from Flashinfer (#8552) 
Co-authored-by: Cheng Wan <cwan@x.ai>
 2025-08-04 03:10:02 -07:00
Yuxuan Zhang
760286e3d3 use fp32 for e_score_correction_bias in GLM-4.5 (#8729) 2025-08-03 10:43:40 -07:00
yhyang201
00da906584 feat: Support DP Attention for step3_vl (#8699) 2025-08-03 19:35:26 +08:00
Varun Vinayak Shenoy
7ed8e51bc3 [fix] Fix divide by zero error for llama4. (#8683) 2025-08-03 00:55:55 -07:00
Wenchen Lo
ea93079b30 model: adapt mllama4 to VisionAttention (#8512) 
Co-authored-by: root <mickjagger19@icloud.com>
 2025-08-02 00:39:40 -07:00
Nicolas Castet
82e6c3a65a Add support for NCCL symmetric memory for TP allreduces (#8238) 2025-08-01 23:30:55 +00:00
Minglei Zhu
2ae95d17e8 Disable tp for shared experts under expert parallelism for GLM4.5 model (#8647) (#8647) 
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-08-01 12:02:35 -07:00
Cheng Wan
6c88f6c8d9 [5/N] MoE Refactor: Update MoE parallelism arguments (#8658) 2025-08-01 01:20:03 -07:00
Baizhou Zhang
e7e5a3050a Update batch size limitation of dsv3_router_gemm kernel to 16 (#8051) 2025-08-01 11:53:31 +08:00
Kaixi Hou
aa4c66b564 [NVIDIA] Enable Flashinfer MoE blockscale fp8 backend for TP MoE (#8450) 
Co-authored-by: kushanam <42385577+kushanam@users.noreply.github.com>
 2025-07-31 19:56:34 -07:00
Faraz
4b04998d38 TRTLLM Gen MLA Decode Kernel Integration (same as #7938) (#8632) 
Signed-off-by: Faraz Khoubsirat <58580514+farazkh80@users.noreply.github.com>
 2025-07-31 16:03:40 -07:00
Ke Bao
8fbcfd0723 Update step3v default config (#8626) 2025-08-01 00:49:26 +08:00
Chang Su
51c38163c1 model: support Step3V (#8583) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: nnnobody-code <nnnobody@foxmail.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Qiaolin-Yu <qy254@cornell.edu>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-07-31 02:41:00 -07:00
Adarsh Shirawalmath
ec5f944271 [Model] Add support for Arcee Foundational Model (#8154) 2025-07-30 10:45:25 -07:00
Cheng Wan
9effeb5bdd Support EPLB in FusedMoE (#8448) 2025-07-29 16:02:41 -07:00
Cheng Wan
9c138a0445 [3/N] MoE Refactor: Simplify DeepEP Output (#8421) 2025-07-28 11:37:17 -07:00
Minglei Zhu
25f73c6cf3 fix GLM4_MOE launch with compressed_tensor quant model (#8456) 2025-07-28 01:31:20 -07:00
Yuxuan Zhang
6d6a8bc278 GLM-4.5 Model Support (#8224) 
Co-authored-by: Lifu Huang <lifu.hlf@gmail.com>
Co-authored-by: Binyao Jiang <byjiang1996@gmail.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-07-27 22:54:07 -07:00
Stefan He
4ad9737045 chore: bump transformer to 4.54.0 (#8416) 
Co-authored-by: Binyao Jiang <byjiang1996@gmail.com>
Co-authored-by: Lifu Huang <lifu.hlf@gmail.com>
 2025-07-27 21:27:25 -07:00
Qiaolin Yu
2810338401 [feat] Support different attention backends for prefill and decode (#6338) 
Co-authored-by: tianqilin.99 <tianqilin.99@bytedance.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-07-28 11:42:29 +08:00
Kaixi Hou
85486b6f6f [NVIDIA] Add Flashinfer MoE blockscale fp8 backend (#8036) 2025-07-27 00:34:41 -07:00
RunningLeon
b7094a5ef1 model: support intern-s1 (#8350) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: zxy <zhou0493@e.ntu.edu.sg>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-07-26 13:48:51 -07:00
Cheng Wan
c0fb25e949 DP Enhancement (#8280) 2025-07-24 21:36:21 -07:00
Ke Bao
e2d66f60c8 Skip llama4 vision module loading when multimodal disabled (#8272) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-07-23 12:41:25 +08:00
Ke Bao
6936be3221 Remve router gemm output dtype conversion (#8204) 2025-07-21 15:37:00 +08:00
Xinyuan Tong
8430bfe3e9 [Refactor] simplify multimodal data processing (#8107) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-07-20 21:43:09 -07:00
Praneth Paruchuri
83c104b188 Feat: Support for Persimmon Model (#7983) 2025-07-19 23:07:47 -07:00