chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
7,725 Commits 1 Branch 0 Tags
9975acf50f220208e2e813c73f322f24af285130
Commit Graph

1339 Commits

Author SHA1 Message Date
Xinyuan Tong
9975acf50f [refactor] Update reasoning parameter to require_reasoning (#14922) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 16:55:59 -08:00
amysaq2023
70758d457e support non-disturbing remote-instance-weight-loader (#13125) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-11 16:45:32 -08:00
Liangsheng Yin
bd7824b24d Minor code style fix for dllm (#14836) 2025-12-11 10:35:42 +09:00
Lianmin Zheng
4285e99da7 [Auto Sync] Update data_parallel_controller.py, detokenizer... (20251209) (#14759) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-12-09 18:38:38 -08:00
ShawnY112358
f077436831 [fix] Fix issues for in-flight weight updates (#14064) 
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
 2025-12-09 18:12:19 -08:00
Liangsheng Yin
cbc7dcdaa7 Re-add the API serving timing metrics. (#14744) 
Signed-off-by: zhanghaotong <zhanghaotong.zht@antgroup.com>
Co-authored-by: zhanghaotong <zhanghaotong.zht@antgroup.com>
 2025-12-10 10:17:48 +09:00
Lianmin Zheng
036e64dafa move multi-item scoring functions in tokenizer manager into a separate file (#14740) 2025-12-09 14:47:06 -08:00
b8zhong
55504df2f7 Add FP8 Blockwise GEMM Backend Flag --fp8-gemm-backend (#14379) 2025-12-09 12:05:56 -08:00
Lianmin Zheng
ab0048793c Revert "[Feat] Add received_time in serving_base" (#14743) 2025-12-09 06:52:27 -08:00
zhanghaotong
fe7f91ef82 [Feat] Add received_time in serving_base (#13432) 
Signed-off-by: zhanghaotong <zhanghaotong.zht@antgroup.com>
 2025-12-09 22:00:24 +09:00
kun-llfl
cef5ba65b1 [Bugfix] Fix environ error in scheduler_runtime_checker_mixin.py (#14461) 
Signed-off-by: Kun(llfl) <i@imux.top>
 2025-12-09 20:33:41 +08:00
Lianmin Zheng
ce4e836be5 Add per-request decode tp size (#14678) 
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
 2025-12-08 20:24:31 -08:00
Yuhao Yang
b9bef31a15 fix: use .get() when accessing strict mem-check env variable (#14657) 2025-12-08 13:25:42 -08:00
Muqi Li
06836ad02a [Reasoning + Structured Output] make reasoning compatible with structured output (#12551) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-08 01:28:39 -08:00
Tiwei Bie
36361adcbf [DLLM] Add initial cuda graph support (#14203) 2025-12-08 14:12:35 +08:00
Nicholas
f57d4fe78e [feat] use cachebuffer to store mm feature to speedup hash (#14386) 2025-12-08 10:35:20 +08:00
khalilzhk
948b6acee8 [BugFix] fix prefixcache performance and accuracy on ascend (#13573) 2025-12-08 02:16:20 +08:00
roikoren755
2ce121a1c3 Enable RadixCache for Mamba2 models (#13584) 2025-12-05 18:23:58 +08:00
Even Zhou
894c0dc57c [NPU][1/N] NPU basic functions refactor and new modelslim quant type (#13359) 2025-12-04 16:15:31 +08:00
Yuhao Yao
77512ae0d7 [bugfix] Fix prefill tbo disabled when --deepep-mode=auto (#14333) 
Co-authored-by: Cheng Wan <wan4ch@gmail.com>
 2025-12-03 01:20:33 -08:00
Lianmin Zheng
ca52ed425f Clean up imports and move files (#14317) 2025-12-02 16:31:54 -08:00
Lianmin Zheng
64092c8b55 [Auto Sync] Rename is_hybrid to is_hybrid_swa (#14252) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: Hanming Lu <hanming@x.ai>
 2025-12-01 23:24:24 -08:00
kun-llfl
106df4eac5 Fix mrope_positions size when req is retracted (#13700) 
Signed-off-by: Kun(llfl) <i@imux.top>
Co-authored-by: Xuchun Shang <xuchun.shang@gmail.com>
 2025-12-02 11:38:20 +08:00
Lianmin Zheng
796d82b107 [Auto Sync] Add max_total_num_tokens metric: Update scheduler_metrics_mixin.py, collector.py (20251202) (#14256) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Dan Zheng <dzheng@x.ai>
 2025-12-01 16:34:34 -08:00
Byron Hsu
edbeaf3b88 [MM][style] rename inputs_embeds to input_embeds for consistency (#14240) 2025-12-01 11:36:51 -08:00
liupeng374
2e8f54e61e [spec-overlap] bugfix for pd disaggregation and npu (#14088) 
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
 2025-12-01 22:58:20 +08:00
fzyzcjy
d815d00248 Tiny call cudaProfilerStart only on first rank in node (#14211) 2025-12-01 18:18:45 +08:00
fzyzcjy
de153cf76a Fix speculative decoding error when retracting (#14180) 2025-12-01 15:30:13 +08:00
fzyzcjy
e8ba5a668c Support profiling only prefill or decode without the other (#14182) 2025-12-01 14:46:30 +08:00
Byron Hsu
0825d7f4c6 [piecewise] Refactor VLM to support input embed buffer and remove external embedder hack (#14155) 2025-11-30 21:43:09 -08:00
PiteXChen
dc7bdc7329 bugfix[schedule]: Excessive preemption occurs when preempting running requests to schedule new prefill requests. (#12494) 
Signed-off-by: CLFutureX <chenyongqyl@163.com>
 2025-11-30 22:29:26 +08:00
fzyzcjy
340c613ab5 Support numactl bind for CPU and memory before process starts (#14156) 2025-11-30 17:00:33 +08:00
fzyzcjy
36b729c2b8 Implement profiler v2 and fix stage mixture bug (#14148) 2025-11-30 16:59:52 +08:00
Lianmin Zheng
155a9e7237 Fix condition for streaming output_ids in tokenizer manager (#13759) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Chang Su <chang.s.su\n@oracle.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-11-29 13:56:15 -08:00
fzyzcjy
4cafc835d3 Super tiny fix typo (#14131) 2025-11-29 21:08:31 +08:00
fzyzcjy
45cf575852 Fix overlap scheduler not take effect when outputing logprobs (#14096) 2025-11-28 18:15:56 +08:00
vipwangerxiao
ab9a46d462 Support configuring the request limit per receiving poll (#14076) 
Co-authored-by: Peng Wang <peng_wang@linux.alibaba.com>
Co-authored-by: Feng Su <225349073+sufeng-buaa@users.noreply.github.com>
 2025-11-28 16:14:21 +08:00
shuwenn
621061f017 [Bugfix] input prompt was not logged (#13936) 2025-11-28 16:00:51 +08:00
Yixin Dong
6350042696 feat: Naive support Spec V2 + Constrained Decoding (#13425) 
Signed-off-by: Ubospica <ubospica@gmail.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-11-27 20:31:46 +08:00
fzyzcjy
25758647b1 Support sanity checking weight consistency especially for RL (#13854) 2025-11-27 20:25:12 +08:00
Jimmy
ab843ced31 [Feat]Add scheduler recv skipper weights to environment configuration (#13855) 2025-11-27 18:16:11 +08:00
ShawnY112358
5155016b56 [feat] update bucketed weights from distributed (#13824) 
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-11-26 15:30:45 -08:00
Vedant V Jhaveri
9dab534b35 fix spec dec request level metrics (#13754) 
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-11-27 01:09:21 +08:00
Michelle Wu
262c3c1fde [Ascend] Support enable-mixed-chunk in non-MLA scenarios (#12491) 
Co-authored-by: ronnie_zheng <zl19940307@163.com>
 2025-11-27 00:00:21 +08:00
Liangsheng Yin
eff6a07c8f Fix get_load API (#13991) 2025-11-26 22:07:51 +08:00
vipwangerxiao
15729dbc8e Use dynamically maintained num_waiting_tokens in get_load() (#13203) 
Signed-off-by: Peng Wang <peng_wang@linux.alibaba.com>
Co-authored-by: Peng Wang <peng_wang@linux.alibaba.com>
 2025-11-26 20:34:45 +08:00
Zehuan Li
21b0582d4b [feature] Initial block diffusion language model support (#12588) 
Co-authored-by: Tiwei Bie <tiwei.btw@antgroup.com>
 2025-11-26 17:57:54 +08:00
ShawnY112358
007c3e234c [feat] support in-flight weight update (#10071) 
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
 2025-11-25 22:03:13 -08:00
Lianmin Zheng
1ab6ce0e62 [Auto Sync] Improve profilers and simplify bench_one_batch_server.py (#13866) 2025-11-25 12:13:31 -08:00
ant-yy
59464dbf15 [Fix]: Further fix the buffer len of future map (#13916) 
Signed-off-by: vito.yy <vito.yy@antgroup.com>
 2025-11-25 18:09:28 +08:00