chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
7,914 Commits 1 Branch 0 Tags
3c8dc448b2aedfcedc0b5f3506083f2ca45a697e
Commit Graph

7914 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Douglas Yang
3c8dc448b2 fix: removing latest-sglang=1 (#15220) 2025-12-16 00:14:48 -08:00
Kenny Yao
5e96beb3e5 Adding tool calling and reasoning parser support for Intern-S1 (#14866) 
Co-authored-by: KennyYao2001 <kennyy@andrew.cmu.edu>
Co-authored-by: Cursor AI <cursor@example.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Changyi Yang <112288487+ChangyiYang@users.noreply.github.com>
 2025-12-16 00:01:58 -08:00
haoyangli-amd
9327482baa [bugfix][quark] Fixed an issue where per_token could not be properly recognized when the token count was 1. (#14415) 2025-12-15 22:54:31 -08:00
Shangming Cai
36fcf71fff [Qwen3-next] Add PD disaggregation support for mamba with extra_buffer (#15180) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: ybyang <10629930+whybeyoung@users.noreply.github.com>
 2025-12-16 14:36:00 +08:00
Xiaoyu Zhang
6292d97135 [diffusion] fix: fix pack qkv opt break tensor parallel (#15225) 2025-12-16 14:33:49 +08:00
Baizhou Zhang
c843419562 Remove duplicate bs=1 in nightly benchmark (#15162) 2025-12-15 22:09:22 -08:00
ゆり
3e4d431a44 [Feature] Add AIME25 dataset support for SGLang simple_eval (#14990) 
Co-authored-by: zkexorability <zkexorability@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-12-15 21:59:40 -08:00
Sai Enduri
538e733e08 [AMD CI] Fix typo. (#15229) 2025-12-15 21:47:01 -08:00
Yibo Cai
22587bc0b4 [BugFix] Fix CPU inference failure (#15231) 2025-12-15 21:32:41 -08:00
Sai Enduri
02d24244e4 [AMD CI] Temporarily disable 2 gpu accuracy test. (#15204) 2025-12-15 20:57:26 -08:00
Muqi Li
1da5cd638f [Bugfix][Tool Call] Add null system prompt to support tool system prompt (#15092) 
Signed-off-by: Muqi Li <muqi1029@gmail.com>
 2025-12-15 19:40:10 -08:00
Tianyu Guo
a9a2cdd8ec Add EPD disaggregation doc (#15224) 2025-12-16 11:22:05 +08:00
XDaoHong
4733fcff1f [Feature] npu support enable_torch_compile for torchair backend (#13410) 
Co-authored-by: ZhengdQin <zhengdqin@gmail.com>
 2025-12-16 09:23:51 +08:00
Yineng Zhang
3ffa260474 chore: update CI_PERMISSIONS (#15212) 2025-12-15 15:09:05 -08:00
Douglas Yang
61f362c694 feature: create docker image from pr branch (#15185) 2025-12-15 10:53:50 -08:00
Kangyan-Zhou
e7157c9b77 Add cache for flashinfer installation (#15153) 2025-12-15 10:37:34 -08:00
Liwansi
30da2f0598 [NPU][eagle3] support qwen eagle3 on NPU (#14820) 2025-12-16 02:25:13 +08:00
Xiaoyu Zhang
4901693110 [diffusion] perf: support FFN pack gate and up proj for Z-Image(#15201) 
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-16 01:18:47 +08:00
ratish
3d484be547 fix(attention): Prevent trtllm_mha auto-selection with eagle3 speculative decoding (#15127) 2025-12-15 16:42:26 +00:00
Xiaoyu Zhang
c0d94440b7 [diffusion] perf: support pack qkv for Z-Image (#15191) 2025-12-16 00:22:24 +08:00
Mick
1dedb63860 [diffusion] chore: minor code cleanups (#15190) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-15 23:57:02 +08:00
Xiaoyu Zhang
7bc8b1532e [diffusion] fix: fix AttributeError in _build_parallelism_config when accessing tp_group.device_group (#15196) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-15 23:41:06 +08:00
roikoren755
9003a4369d Add missing assertion in NemotronH path (#15193) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2025-12-15 06:17:21 -08:00
Simo Lin
3518b33178 [model-gateway] Remove legacy RouterMetrics and Rename SmgMetrics to Metrics and smg_labels to metrics_labels (#15160) 2025-12-15 04:57:42 -08:00
Mick
b098b1ae24 [diffusion] fix: fix video model sp when resolution is not specified (#15047) 
Co-authored-by: Brain97 <Brain97@users.noreply.github.com>
 2025-12-15 20:25:43 +08:00
Lancer
abd3e048f2 [diffusion] fix: fix pytorch non-writable array warning (#15017) 2025-12-15 20:18:54 +08:00
Xiaoyu Zhang
92c29d43ac [diffusion] fix: cache dit with parallel (#15163) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-15 19:15:51 +08:00
Goalina
bf6438142a chore: change npu pr-test a2 runner (#15152) 2025-12-15 19:03:19 +08:00
DarkSharpness
f03bfa4ce3 [Feature] Fuse mrope all in 1 kernel (#14906) 2025-12-15 02:50:55 -08:00
fzyzcjy
89ad390843 Fix num running requests (load) wrong cleared for ongoing requests (#15116) 2025-12-15 17:40:04 +08:00
kk
2ea844ec81 Fused two elementwise kernels for k_nope and k_pe concat (#14862) 
1 TC failure to check, but irrelevant to this code change
 2025-12-15 01:33:33 -08:00
Douglas Yang
1e2d753804 fix: adding date and fixing release name issue (#15174) 2025-12-15 17:11:03 +08:00
blzheng
d16ff357db [CPU] Add Gemma3RMSNorm kernel in sgl-kernel and add ut (#9324) 2025-12-15 00:24:02 -08:00
Douglas Yang
af49e30242 feature: PR wheel (#15170) 2025-12-15 00:22:47 -08:00
Yuhao Yang
01b955ac3d [diffusion] model: support mutli-image input and qwen-image-edit-2509 (#15005) 2025-12-15 16:17:10 +08:00
mmdbhs
16e6bc20b0 fix CompressedTensorsW8A8Int8 min_capability (#13914) 
Co-authored-by: Fan Yin <1106310035@qq.com>
Co-authored-by: Peng Zhang <aniz1905@gmail.com>
 2025-12-15 15:50:39 +08:00
Liangsheng Yin
372507649d Tiny improve summary text in bench_one_batch_server.py (#15158) 2025-12-15 14:59:11 +08:00
Simo Lin
7b9156c773 [model-gateway] add mcp and discovery metrics (#15156) 2025-12-14 22:54:53 -08:00
Douglas Yang
21cfebac65 fix: move ci-bot (#15154) 2025-12-14 22:47:19 -08:00
Lianmin Zheng
702426b06a Fix import warnings (#15144) 2025-12-14 21:25:18 -08:00
Douglas Yang
9e9a61691e ci: adding errors to Github summary (#14778) 2025-12-14 21:08:16 -08:00
Simo Lin
bd9c3a47d6 [model-gateway] Add streaming metrics for harmony gRPC router (#15147) 2025-12-14 20:54:43 -08:00
Simo Lin
1e641ee4a7 [model-gateway] upgrade axum and axum server (#15146) 2025-12-14 20:54:01 -08:00
Simo Lin
fb96669ff9 [model-gateway] Add Layer 3 worker metrics (smg_worker_*) (#15130) 2025-12-14 20:25:46 -08:00
Yuan Luo
3912ee4991 [VLM] feat: support chunked vit attention (#14907) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-15 12:11:02 +08:00
b8zhong
1ab9b8e0a3 Enable TRT AllReduce Fusion by default (#14764) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-14 20:01:49 -08:00
Hanming Lu
e61dabf5e4 [Qwen3-next] support mamba radix cache for overlap scheduler (#14792) 2025-12-14 18:54:16 -08:00
Yineng Zhang
36e7c8c59f docs: update usage (#15142) 2025-12-14 18:42:21 -08:00
Kangyan-Zhou
037c3982af Fix H200 CI by commenting out Warmup Weights and JIT Compilation (#15139) 2025-12-14 18:04:28 -08:00
fzyzcjy
62b3fdae43 Fix cache aware wrong routing caused by incorrect load tracking (#15101) 2025-12-15 10:03:54 +08:00