chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
7,041 Commits 1 Branch 0 Tags
852eb6ce2a76c25cf27d02429c39fc0ee4ee3d0c
Commit Graph

77 Commits

Author SHA1 Message Date
Chen Haozhe
6c2e5fcd91 [feat][Ascend][Mindspore]: support model-impl of mindspore (#9234) 2025-11-19 09:17:47 +08:00
Binyao Jiang
90c18a16cb [GLM4.6v] Required changes for bumping up to transformer 5.x (#13229) 2025-11-18 10:58:00 +08:00
fzyzcjy
33f08a98b0 Tiny refactor condition to requant scale ue8m0 (#13286) 2025-11-15 16:36:00 +08:00
Yang Kaiyong
61bfd9fa9b optm(checkpoint-engine): disable multi-thread loading when update weights (#12374) 
Signed-off-by: Yang Kaiyong <yangkaiyong.yky@antgroup.com>
 2025-11-07 21:30:04 +08:00
yinghui
de0b10cf5c fix: move dummy format loader check before quantization checks (#12532) 2025-11-02 23:41:30 -08:00
b8zhong
77225d602a Use Flashinfer TRT-LLM as Llama 4 compatible MoE backend (#11928) 2025-10-28 10:39:43 -07:00
Mick
770529a731 model: support deepseek-ocr (#11891) 
Co-authored-by: yhyang201 <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Shi Shuai <126407087+shuaills@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-10-24 03:15:17 +08:00
ErvinXie
39c237f02c Add AWQ quantization support for NPU. (#10158) 
Co-authored-by: Alisehen <814073252@qq.com>
Co-authored-by: Yaochen Han <48639761+Alisehen@users.noreply.github.com>
Co-authored-by: Zhengda Qin <zhengdqin@gmail.com>
 2025-10-23 12:08:05 -07:00
Netanel Haber
a98496834b Feature/nano v2 offline modelopt fp8 and nvfp4 (#12018) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
 2025-10-23 11:16:46 -07:00
Liangsheng Yin
6c18addb6f Revert "Support nvidia/NVIDIA-Nemotron-Nano-9B-v2-FP8/NVFP4" (#12015) 2025-10-23 21:27:58 +08:00
Netanel Haber
d6fee73d1f Support nvidia/NVIDIA-Nemotron-Nano-9B-v2-FP8/NVFP4 (#11866) 2025-10-23 17:29:02 +08:00
Christian Bahls
164302c7df Implement BGE-M3 Sparse Embeddings in SGLang (#10869) 
Co-authored-by: Christian Bahls <christian.bahls@planet-ai.de>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-22 13:46:16 -07:00
Zhiyu
80b2b3207a Enable native ModelOpt quantization support (3/3) (#10154) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
 2025-10-21 21:44:29 -07:00
Liangsheng Yin
9d61205dac [lint] improve ruff check (#11922) 
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2025-10-22 11:32:50 +08:00
Lianmin Zheng
cd7e1bd591 Sync code and test CI; rename some env vars (#11686) 2025-10-15 18:37:03 -07:00
Liangsheng Yin
516738b096 Depreate global_server_args_dict (#11528) 2025-10-13 19:34:43 +08:00
Cheng Wan
1bdd010291 Revert "Deprecate global_server_args_dict" (#11520) 2025-10-12 17:40:40 -07:00
Liangsheng Yin
1083e7e3df Deprecate global_server_args_dict (#11331) 2025-10-13 01:20:47 +08:00
Zhiyu
129d299278 Enable native ModelOpt quantization support (2/3) (#9991) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
 2025-10-11 07:48:14 +00:00
Zhiyu
155cbb51f0 Enable native ModelOpt quantization support (1/3) (#7149) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
 2025-10-06 13:24:15 -07:00
Mick
97d966a7f8 ci: make find_local_hf_snapshot_dir more robust (#11248) 2025-10-05 19:50:11 -07:00
Lianmin Zheng
a17e70f5cc Use more general heuristics to set the default value of --mem-fraction-static (#10975) 
Co-authored-by: sglang-bot <sglangbot@gmail.com>
 2025-09-29 10:11:03 -07:00
Mick
43fa9f22bd fix: check if weights are already local before downloading (#11015) 2025-09-28 20:11:33 -07:00
amysaq2023
2bdaf482f9 refactor loading weights from remote instance coding format (#10941) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-09-26 15:25:39 -07:00
ronnie_zheng
095093ee5a [Ascend] optimize Qwen-vl on Ascend (#10556) 
Co-authored-by: wangqihui01 <wangqh10@163.com>
 2025-09-22 17:18:16 -07:00
amysaq2023
30d20ce84f Support loading weights from remote instance (#8215) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
Co-authored-by: Chayenne <74843776+zhaochenyang20@users.noreply.github.com>
 2025-09-12 17:40:22 +08:00
Yi Zhang
30c6e1f569 Qwen3-Next support (#10233) 
Co-authored-by: cao1zhg <114661107+cao1zhg@users.noreply.github.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Binyao Jiang <byjiang1996@gmail.com>
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
Co-authored-by: Lifu Huang <lifu.hlf@gmail.com>
Co-authored-by: qingquansong <ustcsqq@gmail.com>
Co-authored-by: Yaoyao Ding <dingyaoyao.cs@gmail.com>
Co-authored-by: Ke Bao <ISPObaoke@163.com>
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
 2025-09-11 04:11:49 -07:00
Cheng Wan
3fa62da78c [7/N] MoE Refactor: the implementation of new framework (#9269) 2025-09-05 21:09:09 -07:00
wangyu
9f81d741a2 fix: fix MLA for ShardedModelLoader/RemoteModelLoader (#6287) 
Signed-off-by: wangyu <wangyu.steph@bytedance.com>
 2025-08-28 16:10:09 -07:00
fzyzcjy
34e5e11f0f Tiny make device_loading_context more static (#9478) 2025-08-23 02:07:15 -07:00
ichernob
83123f481e [Quantization] Supported w8a8 int8 quantized Gemma3 and Qwen-VL models (#8619) 
Co-authored-by: ronnie_zheng <zl19940307@163.com>
 2025-08-12 13:31:18 -07:00
kk
d4bf5a8524 Support OCP MXFP4 quantization on AMD GPUs (#8255) 
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Hubert Lu <Hubert.Lu@amd.com>
 2025-08-04 18:14:52 -07:00
Even Zhou
99795d61e6 [Bugfix] fix w8a8_int8 load issue (#8308) 
Co-authored-by: ronnie_zheng <zl19940307@163.com>
 2025-07-31 17:30:16 -07:00
Lianmin Zheng
bb0e8a32b5 Clean up server args (#8161) 2025-07-19 11:32:52 -07:00
yilian49
8aa5ae6b04 load draft model fix (#7506) 2025-07-17 21:41:32 -07:00
ronnie_zheng
766392c6bd [feature]Ascend quantization support (#7791) 
Co-authored-by: ichernob <ichernobnn@gmail.com>
Co-authored-by: liupeng <liupeng374@huawei.com>
 2025-07-10 09:17:37 -07:00
Chunyuan WU
1dce6c480f [CPU] support the case where num_attention_heads or intermediate_size is not divisible by the TP size (#6771) 2025-07-03 09:51:38 -07:00
Simon_CQK
a7efbb2757 fix(model loader): use safe_open to prevent file handle leaks. (#7684) 2025-07-01 13:18:35 -07:00
lukec
886d344964 support llama4 eagle3 (#6985) 
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
Co-authored-by: Shenggui Li <somerlee.9@gmail.com>
Co-authored-by: Yingyi Huang <yingyihuang2000@outlook.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
 2025-06-30 22:34:10 -07:00
Lianmin Zheng
22352d47a9 Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632) 
Co-authored-by: Kan Wu <wukanustc@gmail.com>
 2025-06-29 23:16:19 -07:00
xianzhiT
9f1787fa60 Support multi-thread model weight loading (#7277) 2025-06-24 10:39:10 -07:00
Yuhong Guo
e5afb88b1c Support weight loading without mmap (#7469) 2025-06-23 15:13:59 -07:00
Charles Chen
8c16da334e Fix Deepseek R1 0528 FP4 tensor name mismatch issue during weights loading. (#7164) 2025-06-17 11:26:23 -07:00
Baizhou Zhang
2a5f0100e0 Fix GGuf and add back test_gguf.py (#7067) 2025-06-10 21:07:20 -07:00
Marc Sun
37f1547587 [FEAT] Add transformers backend support (#5929) 2025-06-03 21:05:29 -07:00
fzyzcjy
73187152a4 Reland tiny refactor DefaultModelLoader.Source (#6041) 2025-05-17 17:11:20 -07:00
fzyzcjy
6450c1228c Tiny refactor weight loading logic (#5232) 2025-05-08 01:02:56 -07:00
Lianmin Zheng
693723d1f7 Revert "Tiny refactor DefaultModelLoader.Source" (#5825) 2025-04-28 01:18:57 -07:00
fzyzcjy
644ed409d1 Tiny refactor DefaultModelLoader.Source (#5482) 2025-04-28 00:35:51 -07:00
ryang
bc24205b32 Support BNB quantization for llama/mllama (#5038) 
Co-authored-by: Yuhao Yang <yyh073@foxmail.com>
 2025-04-15 18:00:31 -07:00