chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
5,010 Commits 1 Branch 0 Tags
b9eb0d9c2bace993d5aedbfd77460c85830f17be
Commit Graph

40 Commits

Author SHA1 Message Date
Pavani Majety
fcd72bd100 [ModelOpt] Fix Weight Loading for DSR1-FP4 Quantization (#9712) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-08-29 17:13:52 -07:00
blzheng
ebbb75e917 [CPU] Fix TP padding issue on Phi-4 (#8289) 2025-08-17 16:25:26 -07:00
Xiaoyu Zhang
44e86480e8 fuse allreduce and residual_rmsnorm (#8731) 2025-08-11 13:50:53 -07:00
Trevor Morris
c0e84297c2 Use reduce scatter for DP (#8539) 2025-08-06 16:21:26 -07:00
Ying Sheng
c1d2061f97 Add initial support for gpt-oss (#8824) 2025-08-05 13:42:01 -07:00
Nicolas Castet
82e6c3a65a Add support for NCCL symmetric memory for TP allreduces (#8238) 2025-08-01 23:30:55 +00:00
Cheng Wan
15ad6c9086 [1/N] MoE Refactor: refactor select_experts (#7966) 2025-07-19 00:51:15 -07:00
Haohui Mai
d918ab7985 Support NVFP4 quantized dense models on AMD CDNA2/CDNA3 GPUs (#7302) 
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Sai Enduri <saimanas.enduri@amd.com>
 2025-07-18 19:59:39 -07:00
Cheng Wan
49b8777460 Refactor: move all quantization-related code to srt/layer/quantization (#7989) 2025-07-17 00:47:07 -07:00
Xiaoyu Zhang
49a5915f53 [ready b200] fuse allreduce+add_rmsnorm in prepare_attention + mlp module (#7775) 2025-07-10 15:12:39 -07:00
ronnie_zheng
766392c6bd [feature]Ascend quantization support (#7791) 
Co-authored-by: ichernob <ichernobnn@gmail.com>
Co-authored-by: liupeng <liupeng374@huawei.com>
 2025-07-10 09:17:37 -07:00
Chunyuan WU
1dce6c480f [CPU] support the case where num_attention_heads or intermediate_size is not divisible by the TP size (#6771) 2025-07-03 09:51:38 -07:00
Chunyuan WU
9fcc9a80e7 [CPU] refine CPU integration code (#7647) 2025-07-03 09:51:09 -07:00
Chunyuan WU
7eb47b0f3d [CPU] [BF16] Call fused_experts_cpu, weight_packed_linear and bmm_cpu kernel in DeepSeek model (#6641) 
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2025-06-25 01:43:33 -07:00
Baizhou Zhang
2a5f0100e0 Fix GGuf and add back test_gguf.py (#7067) 2025-06-10 21:07:20 -07:00
eigen
8f783c1943 [Model Support] unsloth/Phi-4-mini bnb model (#4982) 
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-04-16 19:58:20 -07:00
Trevor Morris
11d760d56a FP4 weight loading and inference (2/2) (#3972) 2025-04-08 17:26:21 -07:00
Xiaoyu Zhang
04e3ff6975 Support compressed tensors fp8w8a8 (#4743) 2025-03-26 13:21:25 -07:00
Jinyan Chen
f44db16c8e [Feature] Integrate DeepEP into SGLang (#4232) 
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
Co-authored-by: Xuting Zhou <xutingz@nvidia.com>
 2025-03-19 08:16:31 -07:00
aoshen524
588865f0e0 [Feature] Support Tensor Parallelism and Weight Slicing for Lora (#4274) 
Co-authored-by: ShenAo1111 <1377693092@qq.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-03-18 20:33:07 -07:00
HandH1998
0dd6cda288 Apply sgl w8a8 fp8 kernel (#3148) 2025-03-09 00:03:32 -08:00
Lianmin Zheng
ac2387279e Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>
 2025-03-03 00:12:04 -08:00
laixin
1a6e97577a Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3730) 
Co-authored-by: HandH1998 <1335248067@qq.com>
 2025-02-24 05:43:35 -08:00
Liangsheng Yin
8616357a97 Fix deepseek awq v3 (#3450) 2025-02-12 22:09:52 +08:00
Lianmin Zheng
53cef81587 Improve weight loading and code style (#3174) 2025-01-27 03:00:41 -08:00
Yineng Zhang
5dc54f1a62 feat: remove vllm distributed (#2907) 
Co-authored-by: Zhangyi <1109276519@qq.com>
 2025-01-17 22:31:51 +08:00
Yineng Zhang
bf8d07a6f9 feat: patch linear base (#2915) 2025-01-16 18:00:03 +08:00
Lianmin Zheng
6249e4a19e Revert "Integration of TurboMind AWQ" (#2866) 2025-01-13 04:44:39 -08:00
bjmsong
17de02f98d Integration of TurboMind AWQ (#2828) 
Co-authored-by: root <bjmsong@126.com>
 2025-01-13 20:14:16 +08:00
Lianmin Zheng
72c7776355 Fix linear.py and improve weight loading (#2851) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
 2025-01-13 01:39:14 -08:00
Xiaoyu Zhang
f0e15dc6ab [HotFix] fix fp8 scale load failed in tp>1 (#2837) 2025-01-11 14:34:26 +08:00
Lianmin Zheng
8a6906127a Improve linear.py to load sharded weights & remove the dependency of Parameters from vllm (#2784) 
Co-authored-by: SangBin Cho rkooo567@gmail.com
 2025-01-07 23:29:10 -08:00
Zhiyu
287427e2e6 Enable Nvidia's ModelOpt fp8 quantized models (#2535) 2025-01-06 14:54:52 -08:00
HandH1998
53aed988cb Refactor MoE (#2575) 
Co-authored-by: zhyncs <me@zhyncs.com>
 2024-12-26 00:02:14 +08:00
Yineng Zhang
85e1a6f3aa Update model_loader deps and qqq quantization deps (#2220) (#2318) 
Co-authored-by: HandH1998 <1335248067@qq.com>
 2024-12-02 23:22:13 +08:00
Yineng Zhang
8bee20f80b Update vllm to 0.6.3 (#1711) (#1720) 
Co-authored-by: Ke Bao <ISPObaoke@163.com>
 2024-10-19 20:45:41 -07:00
Yineng Zhang
b4408b0d16 feat: update linear deps 1/N (#1305) 2024-09-19 20:53:11 +08:00
Lianmin Zheng
fb1f28cbbb Clean up the comments and names under python/sglang/srt/layers (#1047) 2024-08-12 05:54:37 +00:00
Yineng Zhang
dd7e8b9421 chore: add copyright for srt (#790) 2024-07-28 23:07:12 +10:00
Ying Sheng
2d96da813e refactor model loader [unreachable code]: initial refactor (#655) 2024-07-19 09:27:06 -07:00