chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
10,250 Commits 1 Branch 0 Tags
441045a7bf2496cef0130b03ac88c74eb1cb8e8a
Commit Graph

119 Commits

Author SHA1 Message Date
Alison Shao
a45613f2a6 Revert "[SGL] sync patch: Remove sync points, prefill cudagraph for DP, disable cache reset in mem check (#19190)" (#19581) 
Co-authored-by: Alison Shao <alisonshao@mac.lan>
 2026-02-28 19:46:47 -08:00
Leon Gao
b5a8e4179e [SGL] sync patch: Remove sync points, prefill cudagraph for DP, disable cache reset in mem check (#19190) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
 2026-02-27 18:11:05 -08:00
SoluMilken
07a24f1a38 update pre-commit config (#18860) 2026-02-16 00:18:31 +08:00
Lianmin Zheng
9815ee934c [Auto Sync] Update weight_utils.py (20260212) (#18692) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Dan Zheng <dzheng@x.ai>
 2026-02-12 16:26:05 -08:00
Lianmin Zheng
ce95f203b0 [Auto Sync] Update logits_processor.py (20260209) (#18503) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2026-02-09 14:33:02 -08:00
Zheng Li
27c447653d model: support Qwen3.5 (#18489) 
Co-authored-by: 瑀澈 <yuche.lz@alibaba-inc.com>
 2026-02-10 00:27:59 +08:00
Lianmin Zheng
d4180815a4 Make the functions in logits_processor.py and sampler.py more modular (#17885) 2026-01-28 16:24:23 -08:00
Lianmin Zheng
bc6f0b5ce7 [Auto Sync] Update logits_processor.py, test_logprobs.py (20260124) (#17664) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: yehu-ux <yehu@x.ai>
 2026-01-23 17:57:41 -08:00
Qiaolin Yu
dd93e4451c Support returning customized info (#16144) 2026-01-02 15:32:18 +08:00
Liangsheng Yin
8766a1dd24 [1 / N] Clean up logprob utils (#15509) 2025-12-22 03:12:25 +08:00
Liangsheng Yin
b9ebf0ed63 Clean hidden_states_before_norm (#15485) 2025-12-20 09:38:46 +08:00
Yingchun Lai
160a06cab2 [Feature] Xiaomi MiMo-V2-Flash day0 support (#15207) 
Co-authored-by: 谢学扬 <xiexueyang@xiaomi.com>
Co-authored-by: tz <tangzhen3@xiaomi.com>
Co-authored-by: 李家乐 <lijiale10@xiaomi.com>
Co-authored-by: 张晨 <zhangchen50@xiaomi.com>
Co-authored-by: Shaohui Liu <liushaohui3@xiaomi.com>
Co-authored-by: 王晨 <wangchen77@xiaomi.com>
Co-authored-by: jiangzihan <jiangzihan@xiaomi.com>
Co-authored-by: xiexueyang <xyxie_wangyi@163.com>
Co-authored-by: Linghao Zhang <zhanglinghao@xiaomi.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: JoyFuture <35593546+JoyFuture@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
Co-authored-by: root <root@bj9-ml-g8h20e-k8s-slave106-20251106.alicn.idc.xiaomi.com>
 2025-12-19 11:40:07 +08:00
Liangsheng Yin
0c00220795 tiny unify environ usage (#15335) 2025-12-17 23:31:43 +08:00
Ethan (Yusheng) Su
0c63fb9420 [Feature] Add LoRA support for embedding layers (#14177) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: Beichen-Ma <bm685@cornell.edu>
 2025-12-09 15:53:33 -08:00
Tiwei Bie
36361adcbf [DLLM] Add initial cuda graph support (#14203) 2025-12-08 14:12:35 +08:00
Zehuan Li
21b0582d4b [feature] Initial block diffusion language model support (#12588) 
Co-authored-by: Tiwei Bie <tiwei.btw@antgroup.com>
 2025-11-26 17:57:54 +08:00
Lianmin Zheng
0296f1cdad [Auto Sync] Update activation.py, logits_processor.py, rota... (20251107) (#12853) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-11-07 22:07:51 -08:00
Zhao Chen
d5fa019c36 feat: limit peak memory usage when computing logprobs (#6318) 
Signed-off-by: Zhao Chen <zhaochen.zju@gmail.com>
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
 2025-11-03 23:53:20 -08:00
ykcombat
41efcaeb45 [Feature] PD-Multiplexing Context and Scheduler, lazy import spatial. (#12275) 2025-11-01 00:40:01 +08:00
Yuhong Guo
ab95d35fcb feat: Add Non-intrusive Tensor Dumping for Model Inference (#10566) 2025-10-31 12:04:48 +08:00
Yineng Zhang
ea39952797 Revert "[Feature] PD-Multiplexing Context and Scheduler." (#12267) 2025-10-28 02:00:37 -07:00
ykcombat
05ad28f25e [Feature] PD-Multiplexing Context and Scheduler. (#11592) 2025-10-28 00:54:43 +08:00
fzyzcjy
20bd2271e2 Support true on-policy (#12058) 2025-10-25 10:23:42 +08:00
Lianmin Zheng
cd7e1bd591 Sync code and test CI; rename some env vars (#11686) 2025-10-15 18:37:03 -07:00
Liangsheng Yin
516738b096 Depreate global_server_args_dict (#11528) 2025-10-13 19:34:43 +08:00
Cheng Wan
1bdd010291 Revert "Deprecate global_server_args_dict" (#11520) 2025-10-12 17:40:40 -07:00
Liangsheng Yin
1083e7e3df Deprecate global_server_args_dict (#11331) 2025-10-13 01:20:47 +08:00
Liangsheng Yin
20a6c0a63d Beta spec-overlap for EAGLE (#11398) 
Co-authored-by: Lianmin Zheng <15100009+merrymercy@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
 2025-10-12 11:02:22 +08:00
ykcombat
c8452551ce [Fix] Fix split prefill with fa3. (#11428) 2025-10-11 22:03:28 +08:00
Sundara Raman Ramachandran
53bd00d975 [Generative Score API] Multi-Item scoring with custom attention mask. (#10979) 2025-10-08 18:47:32 -07:00
narutolhy
d17986f8c6 Enable optional FP32 compute for LM Head (#10729) 
Thanks to MiniMax Team and Chenyang Zhao's support.
 2025-09-29 20:45:17 -07:00
Sundara Raman Ramachandran
a360511d7b [Generative Score API] Scoring(Prefill-only) optimizations. (#9748) 2025-09-14 01:57:06 +08:00
Binyao Jiang
9752861002 [Fix] Support qwen3-next MTP+DP (#10392) 2025-09-13 17:45:04 +08:00
kk
78b7465cad Fix GPU fault issue when run dsv3 with dp mode and enable torch-compile (#10361) 
Co-authored-by: wunhuang <wunhuang@amd.com>
 2025-09-12 15:05:51 -07:00
ssshinigami
5dd8c6444b [Bug fix] Fix Gemma 2 and fix Gemma 3 multimodal with bs > 1 on NPU (#9871) 
Co-authored-by: Maksim <makcum888e@mail.ru>
 2025-09-08 01:19:40 -07:00
narutolhy
839c93bd2d feat: add original logprobs to response (#8375) 
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: luhongyu.4869 <luhongyu.4869@bytedance.com>
 2025-08-29 11:43:57 -07:00
Trevor Morris
eff4eb3fdd Add fp4 quantize before all-gather for Flashinfer cutlass MoE DP (max throughput) (#7667) 2025-08-15 22:08:11 -07:00
Cheng Wan
b87aacb5c5 [DP Attention] Refactor: adding some utility functions (#9136) 2025-08-13 21:08:06 -07:00
Cheng Wan
cb099d2095 [CUDA Graph] save cuda graph memory by using next_token_logits_buffer (#8579) 2025-08-03 03:06:47 -07:00
Cheng Wan
e236d8fee8 Save peak memory in logits processor (#8343) 2025-07-26 01:46:42 -07:00
Cheng Wan
c0fb25e949 DP Enhancement (#8280) 2025-07-24 21:36:21 -07:00
Cheng Wan
cb432f1770 saving hidden_states.clone() (#7705) 2025-07-04 20:07:42 -07:00
Chunyuan WU
9fcc9a80e7 [CPU] refine CPU integration code (#7647) 2025-07-03 09:51:09 -07:00
Chunyuan WU
7eb47b0f3d [CPU] [BF16] Call fused_experts_cpu, weight_packed_linear and bmm_cpu kernel in DeepSeek model (#6641) 
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2025-06-25 01:43:33 -07:00
Cheng Wan
e879d8b7a8 [Feature] Comprehensive Hybrid Parallelism Support (#6389) 2025-06-20 14:43:11 -07:00
Neo
0ca594eda9 [FIX]remove redundant code in logits_processor.py (#7079) 2025-06-11 11:49:30 -07:00
Cheng Wan
b2e95f62b4 Fix two issues related to --moe-dense-tp-size=1 (#5657) 
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
Co-authored-by: 颉沆 <xiehang.lsy@alibaba-inc.com>
 2025-05-12 23:51:39 -07:00
Lianmin Zheng
e8e18dcdcc Revert "fix some typos" (#6244) 2025-05-12 12:53:26 -07:00
applesaucethebun
d738ab52f8 fix some typos (#6209) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-05-13 01:42:38 +08:00
Cheng Wan
25c83fff6a Performing Vocabulary Parallelism for LM Head across Attention TP Groups (#5558) 
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
 2025-05-11 23:36:29 -07:00