chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
9,209 Commits 1 Branch 0 Tags
7b22b8ff8a34535b2aa25db642f2d86f82fd22cc
Commit Graph

404 Commits

Author SHA1 Message Date
Kangyan-Zhou
592603d77b Fix flaky streaming logprobs test by handling detokenizer text buffering (#17687) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-01-25 15:09:06 -08:00
Mansoor
bdaa3de075 Add return routed experts to the completions and chat/completions endpoints (#17434) 2026-01-23 12:12:36 -08:00
Yingchun Lai
a5bbcda968 fix: prefer to use max_completion_tokens rather than max_tokens (#17516) 2026-01-21 20:22:20 -08:00
Lianmin Zheng
b74a57a8d9 [Auto Sync] Update detokenizer_manager.py, io_struct.py, mu... (20260120) (#17442) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Wangfan Fu <wangfan@x.ai>
 2026-01-21 14:48:32 -08:00
Qiaolin Yu
76b06bee03 [New Model] GLM4.7-Flash (#17247) 
Co-authored-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Co-authored-by: JustinTong0323 <justinning0323@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2026-01-20 23:44:16 +08:00
ybyang
ebca5879a1 Fix v32 continue_final_message not work (#16567) 2026-01-19 21:45:39 +08:00
Baizhou Zhang
a04675892e Update flashinfer to 0.6.1 (#15551) 2026-01-17 00:48:30 +08:00
shuwenn
8ec160ed46 feature: support uvicorn access log filter(disable logging /metrics) (#15513) 2026-01-15 20:00:06 -08:00
Lianmin Zheng
e7dc85c50b Fix grammar sync across TP ranks (#17100) 
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2026-01-15 18:38:01 -08:00
sglang-bot
000ad42225 chore: bump sgl-kernel version to 0.3.21 (#17075) 
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
 2026-01-15 12:41:17 +08:00
Simo Lin
f091858304 [gRPC] Add GetLoads RPC for comprehensive load metrics (#17087) 2026-01-14 12:05:10 -08:00
Simo Lin
b3a3f51320 [API] Add /v1/loads endpoint for load metrics (#16976) 2026-01-14 09:13:20 -08:00
shuwenn
de94d793ad feat: support qwen3(-VL) rerank scoring&chat template (#16403) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-01-15 00:45:46 +08:00
shuwenn
cd33694585 feat: add --admin-api-key for finer-grained endpoint auth (#15908) 
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
 2026-01-13 20:21:55 -08:00
lg(x)
3a8b44fe89 Update LoRA Weights via Tensor (#16226) 
Co-authored-by: PopSoda2002 <zhouhp.me@gmail.com>
 2026-01-10 17:36:43 +08:00
fzyzcjy
15da3061bc Tiny pass routing key to scheduler processes (#16839) 2026-01-10 10:13:00 +08:00
yuhao
d2ea44f775 VLM: enhance VL embedding model with video input support and revise warm-up strategy (#16635) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-01-08 22:12:01 +08:00
Harish
156d97b219 Fix KeyError when logprobs=false in completions endpoint (#16095) 2026-01-07 15:49:02 -08:00
Yuan Luo
5f3eb377e0 [VLM] Support request level max_dynamic_patch for OpenAI request (#16268) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-04 13:04:43 +08:00
Lianmin Zheng
62f73a8c81 [Auto Sync] Update engine.py (20260102) (#16317) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2026-01-02 17:29:40 -08:00
Feng Su
57d2ba9203 SGLang Tracing: Supports propagating trace headers through sgl.Engine… (#15814) 
Signed-off-by: Feng Su <sufeng@linux.alibaba.com>
 2025-12-31 15:19:53 -08:00
Simo Lin
162d1cf9be [model-gateway] Add classification model support infrastructure (#16061) 
Co-authored-by: Chang Su <chang.s.su@oracle.com>
 2025-12-29 08:34:05 -08:00
Ratish P
ca740a41f3 [model-gateway]: fix grpc embedding test (#15934) 2025-12-27 05:56:54 -05:00
Ratish P
886e038329 [model-gateway]: fix crash in embedding worker health check (#15910) 2025-12-26 12:57:13 -08:00
Kangyan-Zhou
a91e072f33 Use allow auto truncate in the OpenAI API endpoint (#15369) 2025-12-25 18:45:57 -08:00
ratish
5f3a47d8a7 [model-gateway]: add gRPC router embeddings endpoint implementation (#15273) 2025-12-23 09:12:22 -08:00
Lianmin Zheng
5e1a495c65 Improve engine customization interface (#15635) 2025-12-22 14:24:16 -08:00
sglang-bot
34013d9d5a chore: bump sgl-kernel version to 0.3.20 (#15590) 
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
 2025-12-22 12:32:34 -08:00
Junrong Lin
bed301a5ac [Feature] Enable return routed experts (#12162) 
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-12-21 15:16:43 +08:00
mlmz
1f1f05a85e vlm: refactor engine vlm params and support processor output as input (#14091) 
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: zhaochenyang20 <zhaochenyang20@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: BenYao21 <cyao22@asu.edu>
Co-authored-by: minleminzui <minleminzui@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
 2025-12-20 18:31:24 +08:00
Qiaolin Yu
e220da1723 tiny fix sampling seed for completion api (#15498) 2025-12-19 20:32:13 -08:00
Yingchun Lai
160a06cab2 [Feature] Xiaomi MiMo-V2-Flash day0 support (#15207) 
Co-authored-by: 谢学扬 <xiexueyang@xiaomi.com>
Co-authored-by: tz <tangzhen3@xiaomi.com>
Co-authored-by: 李家乐 <lijiale10@xiaomi.com>
Co-authored-by: 张晨 <zhangchen50@xiaomi.com>
Co-authored-by: Shaohui Liu <liushaohui3@xiaomi.com>
Co-authored-by: 王晨 <wangchen77@xiaomi.com>
Co-authored-by: jiangzihan <jiangzihan@xiaomi.com>
Co-authored-by: xiexueyang <xyxie_wangyi@163.com>
Co-authored-by: Linghao Zhang <zhanglinghao@xiaomi.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: JoyFuture <35593546+JoyFuture@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
Co-authored-by: root <root@bj9-ml-g8h20e-k8s-slave106-20251106.alicn.idc.xiaomi.com>
 2025-12-19 11:40:07 +08:00
Jimmy
216067c0cb feat(dsv32): better error handling for DeepSeek-v3.2 encoder (#14353) 2025-12-18 16:34:27 -08:00
Lianmin Zheng
d1f0063262 Clean up __init__ function of the scheduler and event loop for PD (#15298) 2025-12-18 01:35:14 -08:00
Alison Shao
31d48d7f6f Add Ollama-compatible API endpoints + Smart Router (#14376) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-12-16 20:43:38 -08:00
Lianmin Zheng
9d64a7b24f Minor style fixes to the scheduler.py (#15218) 2025-12-16 17:09:44 -08:00
amysaq2023
ccc8f3b266 support non disturbing remote instance weight loader v2 (#14997) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-16 14:39:56 -08:00
Kenny Yao
5e96beb3e5 Adding tool calling and reasoning parser support for Intern-S1 (#14866) 
Co-authored-by: KennyYao2001 <kennyy@andrew.cmu.edu>
Co-authored-by: Cursor AI <cursor@example.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Changyi Yang <112288487+ChangyiYang@users.noreply.github.com>
 2025-12-16 00:01:58 -08:00
Muqi Li
1da5cd638f [Bugfix][Tool Call] Add null system prompt to support tool system prompt (#15092) 
Signed-off-by: Muqi Li <muqi1029@gmail.com>
 2025-12-15 19:40:10 -08:00
danielafrimi
8102e36b5d Add NanoV3 reasoning parser support (#15113) 2025-12-14 12:09:44 -08:00
Xinyuan Tong
96705514bd fix: dpskv32 chat history processing, default drop_thinking to true (#15064) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-13 12:16:04 -08:00
Lianmin Zheng
267170bf1d Clean up server args and engine startup processes (#15015) 2025-12-12 18:46:07 -08:00
Lianmin Zheng
1b5e903480 Refactor of http and engine entrypoints to allow custom override (#14869) 2025-12-12 12:01:08 -08:00
Yineng Zhang
4b7b5af36a Revert several PRs (#14958) 
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-12-12 11:25:12 -08:00
Xinyuan Tong
8bf10e71cd remove dpsk3.2 sys prompt (#14923) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 20:26:13 -08:00
Xinyuan Tong
4885f8b9c1 fix: handle Jinja2 template errors as client errors in OpenAIServingChat (#14748) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 17:06:26 -08:00
Xinyuan Tong
9975acf50f [refactor] Update reasoning parameter to require_reasoning (#14922) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 16:55:59 -08:00
amysaq2023
70758d457e support non-disturbing remote-instance-weight-loader (#13125) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-11 16:45:32 -08:00
Jimmy
5c961756a3 [FIX][DS32]openai protocol: support openai message role: developer (#14304) 2025-12-11 11:34:41 -08:00
fzyzcjy
5eccaf7737 Support HTTP response status code prometheus metrics (#14710) 2025-12-10 22:28:57 +08:00