chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
7,821 Commits 1 Branch 0 Tags
96705514bd7d5ca58e8aa9ced07b4db2615201bc
Commit Graph

364 Commits

Author SHA1 Message Date
Xinyuan Tong
96705514bd fix: dpskv32 chat history processing, default drop_thinking to true (#15064) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-13 12:16:04 -08:00
Lianmin Zheng
267170bf1d Clean up server args and engine startup processes (#15015) 2025-12-12 18:46:07 -08:00
Lianmin Zheng
1b5e903480 Refactor of http and engine entrypoints to allow custom override (#14869) 2025-12-12 12:01:08 -08:00
Yineng Zhang
4b7b5af36a Revert several PRs (#14958) 
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-12-12 11:25:12 -08:00
Xinyuan Tong
8bf10e71cd remove dpsk3.2 sys prompt (#14923) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 20:26:13 -08:00
Xinyuan Tong
4885f8b9c1 fix: handle Jinja2 template errors as client errors in OpenAIServingChat (#14748) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 17:06:26 -08:00
Xinyuan Tong
9975acf50f [refactor] Update reasoning parameter to require_reasoning (#14922) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 16:55:59 -08:00
amysaq2023
70758d457e support non-disturbing remote-instance-weight-loader (#13125) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-11 16:45:32 -08:00
Jimmy
5c961756a3 [FIX][DS32]openai protocol: support openai message role: developer (#14304) 2025-12-11 11:34:41 -08:00
fzyzcjy
5eccaf7737 Support HTTP response status code prometheus metrics (#14710) 2025-12-10 22:28:57 +08:00
Liangsheng Yin
cbc7dcdaa7 Re-add the API serving timing metrics. (#14744) 
Signed-off-by: zhanghaotong <zhanghaotong.zht@antgroup.com>
Co-authored-by: zhanghaotong <zhanghaotong.zht@antgroup.com>
 2025-12-10 10:17:48 +09:00
Lianmin Zheng
18bd8e8d6d Improve CI by trying a warmup before unit tests (#14669) 2025-12-09 15:17:59 -08:00
Lianmin Zheng
ab0048793c Revert "[Feat] Add received_time in serving_base" (#14743) 2025-12-09 06:52:27 -08:00
zhanghaotong
fe7f91ef82 [Feat] Add received_time in serving_base (#13432) 
Signed-off-by: zhanghaotong <zhanghaotong.zht@antgroup.com>
 2025-12-09 22:00:24 +09:00
fzyzcjy
da3dc497b0 Fix dp-aware incompatible with completions and chat completions APIs (#14647) 2025-12-09 13:24:53 +08:00
sglang-bot
2de98010b5 chore: bump sgl-kernel version to 0.3.19 (#14649) 2025-12-08 22:53:08 +08:00
Muqi Li
06836ad02a [Reasoning + Structured Output] make reasoning compatible with structured output (#12551) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-08 01:28:39 -08:00
sglang-bot
d2b42477c7 chore: bump sgl-kernel version to 0.3.18.post3 (#14518) 2025-12-06 13:15:16 -08:00
Baizhou Zhang
42fcf5438f Revert "tiny remove deprecated endpoint call" (#14533) 2025-12-05 23:48:54 -08:00
b8zhong
ec7b2c16d9 tiny remove deprecated endpoint call (#13607) 2025-12-05 09:54:49 -08:00
Simo Lin
7235a7fbe9 [misc] add model arch and type to server info and use it for harmony (#14456) 2025-12-05 06:51:00 -08:00
Yinghai Lu
41429a8c10 [ez] Fix typing (#14473) 2025-12-05 12:23:13 +08:00
Lianmin Zheng
ca52ed425f Clean up imports and move files (#14317) 2025-12-02 16:31:54 -08:00
Eva20150932-atlascloud
7c38eca1e4 feat: DeepSeek new v3.2 encoding (#14249) 
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-02 11:41:05 -08:00
sglang-bot
63b9300f00 chore: bump sgl-kernel version to 0.3.18.post2 (#14244) 2025-12-01 23:14:12 -08:00
fzyzcjy
e8ba5a668c Support profiling only prefill or decode without the other (#14182) 2025-12-01 14:46:30 +08:00
fzyzcjy
340c613ab5 Support numactl bind for CPU and memory before process starts (#14156) 2025-11-30 17:00:33 +08:00
Lianmin Zheng
155a9e7237 Fix condition for streaming output_ids in tokenizer manager (#13759) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Chang Su <chang.s.su\n@oracle.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-11-29 13:56:15 -08:00
fzyzcjy
25758647b1 Support sanity checking weight consistency especially for RL (#13854) 2025-11-27 20:25:12 +08:00
ShawnY112358
5155016b56 [feat] update bucketed weights from distributed (#13824) 
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-11-26 15:30:45 -08:00
ShawnY112358
007c3e234c [feat] support in-flight weight update (#10071) 
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
 2025-11-25 22:03:13 -08:00
fzyzcjy
7130ad3a29 Fix SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION not working (#13961) 2025-11-25 21:58:55 -08:00
sglang-bot
c53e729d45 chore: bump sgl-kernel version to 0.3.18.post1 (#13951) 
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
 2025-11-25 18:14:28 -08:00
Lianmin Zheng
1ab6ce0e62 [Auto Sync] Improve profilers and simplify bench_one_batch_server.py (#13866) 2025-11-25 12:13:31 -08:00
Zhi Yiliu
a95a38078b [Fix] Fix uvloop get_event_loop() is not suitable for 0.22.x (#13612) 
Signed-off-by: lzy <tomlzy213@gmail.com>
Co-authored-by: lzy <tomlzy213@gmail.com>
 2025-11-25 01:20:00 +08:00
Baizhou Zhang
04b52fa8d6 [chore]Upgrade flashinfer to 0.5.3 (#13751) 2025-11-23 23:38:36 -08:00
Lianmin Zheng
90a0133515 [Auto Sync] Update http_server.py, io_struct.py, scheduler_... (20251120) (#13679) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Zhuqi Li <zhli@x.ai>
 2025-11-20 23:17:15 -08:00
ishandhanani
b537ac0d1f Fix ZMQ bind error on non-zero rank nodes when using SGLANG_BLOCK_NONZERO_RANK_CHILDREN=0 (#13686) 2025-11-20 22:54:41 -08:00
sglang-bot
bfaf0b8607 chore: bump sgl-kernel version to 0.3.17.post2 (#13570) 2025-11-19 14:02:57 -08:00
Binyao Jiang
26ca07469b [GLM4.6v] Relax the constraint of non-user role chat completion message schema for new GLM-v release (#13258) 2025-11-18 10:58:17 +08:00
Binyao Jiang
90c18a16cb [GLM4.6v] Required changes for bumping up to transformer 5.x (#13229) 2025-11-18 10:58:00 +08:00
Simo Lin
25acbbc612 Remove verbs from GET endpoint paths to follow REST standards (#13273) 2025-11-17 12:34:36 -08:00
kebyn
15db5497d3 [feature] Custom base path on FastAPI server (#5879) 
Co-authored-by: lianhu.yin <lianhu.yin@nio.com>
Co-authored-by: kebyn <kebyn@kebyn.cc>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-11-17 22:40:09 +08:00
sglang-bot
1ca205f6da chore: bump sgl-kernel version to 0.3.17.post1 (#13358) 2025-11-15 19:11:41 -08:00
Lianmin Zheng
db7299aa30 [Fix] Register custom ops only if they exist (#13321) 2025-11-15 17:48:17 -08:00
Zilin Zhu
f0b5ccf5f5 [RL] Allow bypassing /health check (#13320) 2025-11-15 16:33:54 +08:00
Yineng Zhang
f8d3d80f63 chore: bump flashinfer v0.5.2 (#13242) 2025-11-14 02:47:09 -08:00
fzyzcjy
ed1d18d472 Tiny fix update version logic location (#12620) 2025-11-14 17:33:36 +08:00
billishyahao
ee3e337c62 [feat] make warmup timeout configurable through SGLANG_WARMUP_TIMEOUT (#13243) 2025-11-14 10:14:37 +08:00
Binyao Jiang
9b41f31a66 Use 32x32 black image for VLM server warmup and bring glm4.1v back to UT (#13222) 2025-11-13 14:21:43 -08:00