chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
7,254 Commits 1 Branch 0 Tags
70c6f95107bd2f2f567fcc1ab679ef6800d41da5
Commit Graph

43 Commits

Author SHA1 Message Date
Cheng Wan
6d0e0b9bfc [11/N] MoE Refactor: Simplifying SBO Implementation with Dispatcher Hooks (#13327) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-11-21 01:11:37 -08:00
Liangsheng Yin
61c7fe7aed Minor code cleanup / improvement for PREBUILT_EXTEND mode (#12948) 2025-11-10 11:59:54 +08:00
Liangsheng Yin
4f65a64666 Refactor / Unify event loop across PD-Disagg, Overlap, DP-Attn cases (#12839) 
Co-authored-by: cctry <17473714+cctry@users.noreply.github.com>
 2025-11-10 00:42:50 +08:00
Liangsheng Yin
243ea585fc [DP-Attn] Clarify MLP sync / idle batch preparation logic (#12843) 2025-11-08 23:23:14 +08:00
Cheng Wan
bfc3b3f786 [9/N] MoE Refactor: cleanup dispatcher interfaces (#11847) 2025-10-20 10:11:46 -07:00
Cheng Wan
5b214b50b6 [Refactor] move deep_gemm_wrapper out of quantization (#11784) 2025-10-17 18:57:54 -07:00
Xun Sun
a40229f6f8 [1/N] Introduce Mooncake Backend and Mooncake EP to Support Elastic EP (#10423) 
Co-authored-by: Hank Han <hanhan7630@outlook.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-10-14 19:40:54 -07:00
Liangsheng Yin
516738b096 Depreate global_server_args_dict (#11528) 2025-10-13 19:34:43 +08:00
Cheng Wan
1bdd010291 Revert "Deprecate global_server_args_dict" (#11520) 2025-10-12 17:40:40 -07:00
Liangsheng Yin
1083e7e3df Deprecate global_server_args_dict (#11331) 2025-10-13 01:20:47 +08:00
Liangsheng Yin
20a6c0a63d Beta spec-overlap for EAGLE (#11398) 
Co-authored-by: Lianmin Zheng <15100009+merrymercy@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
 2025-10-12 11:02:22 +08:00
Sundara Raman Ramachandran
53bd00d975 [Generative Score API] Multi-Item scoring with custom attention mask. (#10979) 2025-10-08 18:47:32 -07:00
fzyzcjy
efbc687c28 Support DeepSeek V3.2 Exp (#11061) 
Co-authored-by: Stefan He <11166516+hebiao064@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <95566987+hnyls2002@users.noreply.github.com>
Co-authored-by: Baizhou Zhang <56809903+fridge003@users.noreply.github.com>
Co-authored-by: DarkSharpness <76582120+darksharpness@users.noreply.github.com>
Co-authored-by: ZhengdQin <46387172+zhengdqin@users.noreply.github.com>
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Zhengda Qin <zhengdqin@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-10-06 00:24:15 -07:00
Liangsheng Yin
73d4a5f879 Organize spec-related data structures (#10735) 2025-10-01 09:45:30 +08:00
wejoncy
a9050b5c39 [bugfix]Add empty_context import to two_batch_overlap.py (#10964) 2025-09-28 01:26:20 -07:00
Cheng Wan
295895120d [6/N] MoE Refactor: Cleanup MoE-related configs (#8849) 2025-08-14 21:14:53 -07:00
Cheng Wan
b87aacb5c5 [DP Attention] Refactor: adding some utility functions (#9136) 2025-08-13 21:08:06 -07:00
Chaitanya Sri Krishna Lolla
323bc2f51a Enable TBO on ROCm (#8329) 2025-08-09 01:59:55 -07:00
PGFLMG
b7cd743038 [Feat] QWen-1M context support[2/2]: Update block sparse attention backend (#5949) 2025-08-06 23:49:36 -07:00
HouseWest
ca47e24f5d [Feature] improve TBO: two chunk overlap (#8144) 2025-08-05 21:11:01 -07:00
Baizhou Zhang
f2d68ded6d Rename lora_path to lora_id in batches (#8437) 2025-08-03 21:08:28 -07:00
Cheng Wan
cb099d2095 [CUDA Graph] save cuda graph memory by using next_token_logits_buffer (#8579) 2025-08-03 03:06:47 -07:00
Cheng Wan
6c88f6c8d9 [5/N] MoE Refactor: Update MoE parallelism arguments (#8658) 2025-08-01 01:20:03 -07:00
Cheng Wan
9c138a0445 [3/N] MoE Refactor: Simplify DeepEP Output (#8421) 2025-07-28 11:37:17 -07:00
Cheng Wan
c0fb25e949 DP Enhancement (#8280) 2025-07-24 21:36:21 -07:00
ZhichenJiang
f4674df646 support idle batch for TBO (#8233) 2025-07-24 20:43:52 -07:00
Xiaoze Fan
570d33437b [Feature] Layer-wise Prefill (#7634) 
Signed-off-by: jason-fxz <jason341132@qq.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-07-17 01:57:46 +08:00
Shangming Cai
d379bda4fa [Bugfix] Fix two batch overlap with auto DeepEP Dispatch (#7853) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-07-08 02:49:32 -07:00
fzyzcjy
a291439a59 Support logprobs in two-batch overlap (#7709) 2025-07-05 19:05:32 -07:00
Cheng Wan
8fc910db03 DP Attention with Auto DeepEP Dispatch (#7222) 2025-07-05 01:54:24 -07:00
Qiaolin Yu
41650b0d70 feat: support compatibility between MTP and two-batch-overlap (#7225) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-06-27 01:10:27 -07:00
Qiaolin Yu
b8df43ab9c Fix gathered_buffer issues in tbo (#7531) 2025-06-25 14:42:21 -07:00
fzyzcjy
b4c41f7276 Refactor DeepGEMM integration (#7150) 2025-06-13 20:41:03 -07:00
fzyzcjy
2fc1299562 Remove unnecessary kernels of num_token_non_padded (#6965) 2025-06-08 05:09:17 -07:00
Lianmin Zheng
608668e143 Slightly improve the sampler to skip unnecessary steps (#6956) 2025-06-08 03:18:54 -07:00
fzyzcjy
31589e177e Speed up when having padding tokens two-batch overlap (#6668) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-05-28 16:00:58 -07:00
fzyzcjy
ae6a5b2950 Minor refactor two-batch overlap (#6682) 2025-05-28 15:54:17 -07:00
Yi Zhang
b18416fbf8 Fix qwen3 tbo/dp-lm-head (#6652) 2025-05-27 00:38:27 -07:00
fzyzcjy
32cd707002 Support TP in attention for two batch overlap (#6634) 2025-05-26 20:28:12 -07:00
Yi Zhang
f9bab3d591 qwen3moe support two batch overlap (#6598) 2025-05-25 23:08:16 -07:00
fzyzcjy
93e53f6e0b Logging and minor fixes to two batch overlap and EPLB (#6595) 2025-05-25 22:36:40 -07:00
fzyzcjy
a191a0e47c Improve performance of two batch overlap in some imbalanced cases (#6593) 2025-05-25 22:36:18 -07:00
fzyzcjy
0d47788025 Support overlapping two batches (#4068) 2025-05-24 17:39:07 -07:00