chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
8,588 Commits 1 Branch 0 Tags
3aa11ca722ce25d6d1f291636b3bcdd7b3f43dd9
Commit Graph

4702 Commits

Author SHA1 Message Date
Ke Bao
3aa11ca722 Remove hybrid_kvcache_ratio in server args (#16399) 2026-01-06 13:13:13 +08:00
zhangheng
2d02c150dc [3/N][Sparse With Hicache]: Init sparse coordinator (#16086) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-01-05 17:51:04 -08:00
ShirakSyouya
ce694b2b9e Fix the problem where Qwen3VL raises an "object has no attribute 'mod… (#15677) 2026-01-06 08:45:13 +08:00
Lianmin Zheng
6c0fb189f8 [Auto Sync] Update tokenizer_manager.py (20260105) (#16477) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Wangfan Fu <wangfan@x.ai>
 2026-01-05 14:06:03 -08:00
Binyao Jiang
9a9f996f8d [FP8] Fix weight_scale shape to match with x_scale shape for per-tensor quant under torch.compile (#16356) 2026-01-05 13:55:26 -08:00
Yi Zhang
a3914e3b3f [Bugfix] fix some memory computation bugs for qwen3next with mtp (#16138) 2026-01-05 23:24:21 +08:00
Raayan Dhar
4397cda7dc feat: raise error in PD when page sizes are mismatched (#14474) 
Signed-off-by: Raayan Dhar raayan.dhar@gmail.com <raayan.dhar@gmail.com>
Signed-off-by: raayandhar <raayan.dhar@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-01-05 19:54:01 +08:00
Lianmin Zheng
c35eb0fdee [Auto Sync] Update loader.py (20260105) (#16478) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: xiuyuli-xai <xiuyu@x.ai>
 2026-01-05 01:34:30 -08:00
fzyzcjy
7f35c46efb Tiny add sglang:http_requests_active metric (#16479) 2026-01-05 17:20:31 +08:00
DarkSharpness
da2f8cc33f [minor] reduce 1 unnecessary add (#16474) 2026-01-05 01:00:56 -08:00
fzyzcjy
4d737db857 Tiny fix prefill delayer not support non-fcfs schedule policy (#16471) 2026-01-05 16:28:54 +08:00
fzyzcjy
9d6029fb92 Fix TokenizerManager bottleneck for offline generation (#16456) 2026-01-05 16:28:25 +08:00
Netanel Haber
bebd625ba1 EVS Framework: Support NemotronH_Nano_VL_V2 (#14051) 2026-01-05 16:18:07 +08:00
Lianmin Zheng
012dc5866d [Auto Sync] Update scheduler.py (20260104) (#16424) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: Hanming Lu <hanming@x.ai>
Co-authored-by: Jie Ren <no-replay@github.com>
 2026-01-04 23:32:56 -08:00
Yuhao Yang
10174e1114 Revert "[grpc] update api to scheduler in grpc request manager" (#16387) 2026-01-04 22:05:39 -08:00
Yuhao Yang
2138ff48c6 Revert "[FEAT] optimize tensor zmq transfer for multimodal inputs" (#16386) 2026-01-04 22:05:26 -08:00
yudian0504
561a3e04f6 remove redundant mamba_cache clearing actions (#16180) 2026-01-05 13:02:07 +08:00
YAMY
12cde0df99 [SPEC_V2] Fix Acclen drop when enabling DP Attention for Spec-Overlap (#16310) 2026-01-05 11:39:23 +08:00
fzyzcjy
1e7b326482 Super tiny fix main code (#16432) 2026-01-05 10:34:25 +08:00
Mick
9a8ba3c189 [diffusion] feat: support warmup with resolutions (#16330) 2026-01-05 10:16:26 +08:00
Xiaoyu Zhang
0fee6bc632 [JIT kernel] Apply jit per_tensor_quant_fp8 kernel (#15836) 2026-01-05 10:15:00 +08:00
fzyzcjy
12df16607b Tiny speed up kimi detokenizer by 10x (#16427) 2026-01-05 09:12:05 +08:00
Ke Bao
76bc07a335 Move swa memory pool to a seperate file (#16347) 2026-01-04 22:39:30 +08:00
Ke Bao
b328cd20bb Skip local attn init metadata for mimo swa model (#16349) 2026-01-04 22:38:36 +08:00
Douglas Yang
16e00651dd fix: health check adding test delays (#16381) 2026-01-04 00:28:32 -08:00
fzyzcjy
1b2b95d8d1 Super tiny cleanup unused function (#16376) 2026-01-04 15:08:12 +08:00
Yuan Luo
5f3eb377e0 [VLM] Support request level max_dynamic_patch for OpenAI request (#16268) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-04 13:04:43 +08:00
fzyzcjy
d7aa0ce72f Support multiple engines for router simulation in schedule simulator (#16368) 2026-01-04 11:46:05 +08:00
fzyzcjy
e797f0c570 Support offline generation scenario for prefill delayer (#16363) 2026-01-04 11:05:10 +08:00
fzyzcjy
5d4b7c78bf Fix memory leak in prefill delayer (#16358) 2026-01-04 10:11:18 +08:00
fzyzcjy
24e116ef6a Enhance schedule simulator and add several e2e scenarios (#16357) 2026-01-04 10:02:38 +08:00
fzyzcjy
8ca9597057 Tiny support sticky routing algorithm in schedule simulator (#16355) 2026-01-04 08:36:52 +08:00
fy
25fa2ac290 Convert cu_seqlens to CPU for npu_flash_attention_unpad operator (#15434) 2026-01-04 08:16:42 +08:00
fzyzcjy
ef5ac6f01e Support shared prefix (gsp) in schedule simulator (#16353) 2026-01-04 08:03:16 +08:00
fzyzcjy
2337b1bbb0 Refactor and fix prefill delayer (scheduler enhancer) (#16269) 2026-01-04 07:15:06 +08:00
Chang Su
7bc13c906c [grpc] update api to scheduler in grpc request manager (#16350) 2026-01-03 10:54:31 -08:00
fzyzcjy
87ef05e2e1 Support simple schedule simulator (#16344) 2026-01-03 22:05:40 +08:00
fzyzcjy
b65c9889a0 Fix incorrect running batch size in prefill stats (#15941) 2026-01-03 20:49:54 +08:00
fzyzcjy
c7c0d97fc6 Tiny support whitelisted headers in request logging (#16342) 2026-01-03 20:10:58 +08:00
shengzhaotian
6bc5a52fd2 [NPU] Adapt qwen3-next W8A8 on NPU (#16164) 2026-01-03 19:41:20 +08:00
fzyzcjy
7f2fa2167b Tiny add --log-requests-target (#16338) 2026-01-03 17:28:27 +08:00
Siyuan Chen
9a414b164c [Performance] Optimze the performance of Qwen25VL (#15640) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-01-02 23:15:36 -08:00
Teng Ma
bdde949619 [HiCache] Add PP Support with suffix pp rank (#15175) 
Co-authored-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: ybyang <10629930+whybeyoung@users.noreply.github.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-01-03 13:49:21 +08:00
fzyzcjy
9821fae5c6 Tiny add explanations to realtime token metric labels (#16009) 2026-01-03 12:36:55 +08:00
siyu
078d96213a [FEAT] optimize tensor zmq transfer for multimodal inputs (#13592) 
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
 2026-01-03 12:23:23 +08:00
b8zhong
74a166cb86 [Fix] Only add SM90 and SM100 to check for auto-enabling TRT Allreduce Fusion (#16283) 2026-01-03 11:43:17 +08:00
Baizhou Zhang
bb23a8fe77 [Tiny]Remove progress bar for fp8 ue8m0 quant when unneeded (#16177) 2026-01-03 10:54:12 +08:00
sunxxuns
8b869e326c [AMD] feat: add DLLM support for AMD GPUs with LLaDA2 testing (#15560) 2026-01-03 10:41:11 +08:00
Lianmin Zheng
62f73a8c81 [Auto Sync] Update engine.py (20260102) (#16317) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2026-01-02 17:29:40 -08:00
Ke Bao
7c1b4b1c4c Support swa allocator page size > 1 (#16296) 2026-01-03 08:49:47 +08:00