chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
9,506 Commits 1 Branch 0 Tags
d0c39bc2193c3e28eee94018340d32029541f0a3
Commit Graph

118 Commits

Author SHA1 Message Date
Alison Shao
d0c39bc219 Fix cross-container HF download race condition in CI (#18328) 2026-02-05 21:01:41 -08:00
Mohammad Miadh Angkad
6f6b9c6e42 [Perf] Use safetensors load_file in multithread loader (#18124) 2026-02-02 23:21:13 -08:00
Alison Shao
28e2340725 Fix HF hub race condition in CI by coordinating model downloads across TP ranks (#17787) 
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
 2026-02-02 14:57:45 -08:00
Yuhao Yang
980d2936cd model: support Step-3.5-Flash (#18084) 
Co-authored-by: ltd0924 <ltd0924@sina.com>
 2026-02-03 00:40:07 +08:00
jianan-gu
336dc4579e [CPU] Optimize Qwen3-next model on CPU (#12525) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
Co-authored-by: Fan Yin <1106310035@qq.com>
 2026-01-29 22:03:58 -08:00
baonudesifeizhai
84ab611af8 model: support DeepSeek-OCR-2 (#17897) 2026-01-30 09:49:51 +08:00
Kangyan-Zhou
c3bf53c7c1 Fix ci weight validation logic to check the safetensor completeness (#17917) 2026-01-29 13:00:42 -08:00
Ziang Li
3c9cc44ff5 Add mxfp8 support for online quantization, Triton dense linear, and CUTLASS MoE (#17449) 2026-01-29 21:33:57 +08:00
Zhongdongming Dai
1b56a886bb [chore]: improve time tracing of model loading process (#15426) 
Co-authored-by: Michael Shin <mmshin@nvidia.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
 2026-01-26 19:04:25 -08:00
Alison Shao
9121f22656 Add PyTorch .bin file validation to CI weight validation (#17533) 2026-01-24 19:18:15 -08:00
b8zhong
2ff0880a0e [Fix] GLM 4.7 + NVFP4 + MTP (#17166) 2026-01-21 21:34:18 +08:00
Hudson Xing
21ee597e4a ci: enable offline mode when local cache is complete to avoid HF Hub … (#16121) 2026-01-15 20:15:33 -08:00
fxmarty-amd
5af84c8af5 [AMD][Quantization] Add int4fp8_moe online quantization on ROCm (#7392) 
Co-authored-by: Dehua Tang <dehtang@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: YC Tseng <yctseng@amd.com>
 2026-01-14 01:44:40 -08:00
Lianmin Zheng
a4825ed588 Fix kernel type annotations for fp8 quant and logging (#16994) 2026-01-13 18:14:32 -08:00
fzyzcjy
d54f0a10b4 Support bitwise weight checksum verifier (#16729) 2026-01-08 20:19:25 +08:00
Alison Shao
63cc97f4ef ci: migrate 2-GPU tests to test/registered/ (#16529) 2026-01-07 20:28:16 -08:00
Lianmin Zheng
c35eb0fdee [Auto Sync] Update loader.py (20260105) (#16478) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: xiuyuli-xai <xiuyu@x.ai>
 2026-01-05 01:34:30 -08:00
Kangyan-Zhou
fc643ffbc9 Download missing shards in model weights files when not in CI (#16211) 2025-12-31 20:42:34 +08:00
Lianmin Zheng
e6d5a213ad Fix metrics (#15998) 2025-12-28 05:03:49 -08:00
Alison Shao
0e536600e8 Refactor: separate CI-specific weight validation into dedicated module (#15216) 
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
 2025-12-27 20:50:39 -08:00
Teng Ma
d7301c89ba [Feature] support fastsafetensors (#15091) 
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Xuchun Shang <xuchun.shang@gmail.com>
 2025-12-23 22:33:56 +08:00
Yingchun Lai
160a06cab2 [Feature] Xiaomi MiMo-V2-Flash day0 support (#15207) 
Co-authored-by: 谢学扬 <xiexueyang@xiaomi.com>
Co-authored-by: tz <tangzhen3@xiaomi.com>
Co-authored-by: 李家乐 <lijiale10@xiaomi.com>
Co-authored-by: 张晨 <zhangchen50@xiaomi.com>
Co-authored-by: Shaohui Liu <liushaohui3@xiaomi.com>
Co-authored-by: 王晨 <wangchen77@xiaomi.com>
Co-authored-by: jiangzihan <jiangzihan@xiaomi.com>
Co-authored-by: xiexueyang <xyxie_wangyi@163.com>
Co-authored-by: Linghao Zhang <zhanglinghao@xiaomi.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: JoyFuture <35593546+JoyFuture@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
Co-authored-by: root <root@bj9-ml-g8h20e-k8s-slave106-20251106.alicn.idc.xiaomi.com>
 2025-12-19 11:40:07 +08:00
Liangsheng Yin
0c00220795 tiny unify environ usage (#15335) 2025-12-17 23:31:43 +08:00
LHXuuu
712f44ee2b fix qwenvl compressed tensors quantization weight loader (#11914) 
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: Peng Zhang <aniz1905@gmail.com>
 2025-12-17 18:01:43 +08:00
amysaq2023
ccc8f3b266 support non disturbing remote instance weight loader v2 (#14997) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-16 14:39:56 -08:00
Yineng Zhang
4b7b5af36a Revert several PRs (#14958) 
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-12-12 11:25:12 -08:00
Peng Zhang
8f5adac8c6 [RL] refactor flash rl weight reload in sglang (#14870) 
Co-authored-by: eternally-z <zzywzj@gmail.com>
 2025-12-12 12:50:47 +08:00
amysaq2023
70758d457e support non-disturbing remote-instance-weight-loader (#13125) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-11 16:45:32 -08:00
Yinghai Lu
b05b346a13 [loader] enable private loader (#14620) 2025-12-11 11:49:46 -08:00
Alison Shao
f85460fb19 Avoid deleting entire cache for missing shards (#14754 follow-up) (#14853) 2025-12-11 01:17:04 -08:00
Alison Shao
b6523a4f72 fix: restrict cache validation behaviors to CI only (#14849) 2025-12-10 16:03:53 -08:00
Peng Zhang
21028b5507 [RL] support weight reload for low-bit rollout (#9650) 
Co-authored-by: Hecate0821 <hec4te0821@gmail.com>
Co-authored-by: eternally-z <zzywzj@gmail.com>
Co-authored-by: Wilboludriver <wilbolu@outlook.com>
Co-authored-by: Wilbolu <81792854+Wilboludriver@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
 2025-12-10 15:44:01 +08:00
Alison Shao
01835998e1 fix: race condition between validation and download locks (#14761) 2025-12-09 20:36:54 -08:00
Alison Shao
b988c18eae Fix safetensors validation to catch corruption after download (#14465) 2025-12-05 16:04:00 -08:00
Alison Shao
662809874c Add Mistral Large 3 to nightly CI tests (#14459) 2025-12-05 23:16:27 +08:00
Xinyuan Tong
6d37e70883 ministral3 (#14251) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Yueming Yuan <yy28@illinois.edu>
 2025-12-04 14:31:26 -08:00
alisonshao
80518bea65 Fix validation to detect missing model files before loading (#14253) 2025-12-03 11:36:07 -08:00
alisonshao
25a6be4930 Fix duplicate download log messages in multi-process environment (#14299) 2025-12-02 09:33:18 -08:00
Zhiyu
079b173853 Fix a distributed initialization error (#13843) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
 2025-12-01 15:10:05 -08:00
alisonshao
a223402ffb Add adapter_model.safetensors to corruption validation for LoRA (#14022) 2025-11-26 14:52:46 -08:00
alisonshao
94ae816f29 Fix model weights validation with automatic cache cleanup (#13729) 2025-11-22 11:40:00 +08:00
Chen Haozhe
6c2e5fcd91 [feat][Ascend][Mindspore]: support model-impl of mindspore (#9234) 2025-11-19 09:17:47 +08:00
Binyao Jiang
90c18a16cb [GLM4.6v] Required changes for bumping up to transformer 5.x (#13229) 2025-11-18 10:58:00 +08:00
fzyzcjy
33f08a98b0 Tiny refactor condition to requant scale ue8m0 (#13286) 2025-11-15 16:36:00 +08:00
Yang Kaiyong
61bfd9fa9b optm(checkpoint-engine): disable multi-thread loading when update weights (#12374) 
Signed-off-by: Yang Kaiyong <yangkaiyong.yky@antgroup.com>
 2025-11-07 21:30:04 +08:00
yinghui
de0b10cf5c fix: move dummy format loader check before quantization checks (#12532) 2025-11-02 23:41:30 -08:00
b8zhong
77225d602a Use Flashinfer TRT-LLM as Llama 4 compatible MoE backend (#11928) 2025-10-28 10:39:43 -07:00
Mick
770529a731 model: support deepseek-ocr (#11891) 
Co-authored-by: yhyang201 <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Shi Shuai <126407087+shuaills@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-10-24 03:15:17 +08:00
ErvinXie
39c237f02c Add AWQ quantization support for NPU. (#10158) 
Co-authored-by: Alisehen <814073252@qq.com>
Co-authored-by: Yaochen Han <48639761+Alisehen@users.noreply.github.com>
Co-authored-by: Zhengda Qin <zhengdqin@gmail.com>
 2025-10-23 12:08:05 -07:00
Netanel Haber
a98496834b Feature/nano v2 offline modelopt fp8 and nvfp4 (#12018) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
 2025-10-23 11:16:46 -07:00