chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
5,386 Commits 1 Branch 0 Tags
93f75778beb9a0387c7f3e0f974b39addeea8e94
Commit Graph
78 Commits
Author SHA1 Message Date
penguin_wwy 93f75778be [RL] Add destroy process group api (#9979) 2025-09-19 00:31:56 +08:00
Liangsheng Yin fa5d0bf6a5 Remove wrong imports from sglang.python (#10493) 2025-09-15 22:12:21 -07:00
Liangsheng Yin 305c9e8c2d [4/N]DP refactor: support watching mode get_load and shortest queue strategy (#10201) 2025-09-15 10:06:08 +08:00
Feng Su 4c21b09074 [Feature] Sglang Tracing: Fine-Grained Tracking for Request Latency - Part 1 (#9962) 
Signed-off-by: Feng Su <sufeng@linux.alibaba.com>
Signed-off-by: Huaixin Chang <changhuaixin@linux.alibaba.com>
Signed-off-by: Peng Wang <rocking@linux.alibaba.com>
 2025-09-15 02:08:02 +08:00
amysaq2023andChayenne 30d20ce84f Support loading weights from remote instance (#8215) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
Co-authored-by: Chayenne <74843776+zhaochenyang20@users.noreply.github.com>
 2025-09-12 17:40:22 +08:00
ybyang 6d40308905 Revert add mainprocess's proctitle (#10351) 2025-09-12 16:48:30 +08:00
Liangsheng YinandHuang Long 72f9fc5f11 Monkey patch uvicorn multi worker is_alive timeout (#10159) 
Co-authored-by: Huang Long <121648372+llll114@users.noreply.github.com>
 2025-09-08 17:43:23 +08:00
Liangsheng Yin e719bb0e84 [1/2] Refactor multi-tokenizer manager (#10074) 2025-09-07 19:13:34 +08:00
Liangsheng Yin 6e95f5e5bd Simplify Router arguments passing and build it in docker image (#9964) 2025-09-05 12:13:55 +08:00
Liangsheng Yinandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 5dfcd6c207 add proctitle for tokenizers (#9952) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-03 13:31:38 +08:00
Lianmin Zheng 60e37f8028 Move parsers under a single folder (#9912) 2025-09-02 18:25:04 -07:00
ybyangHuang LonghuanglongShangming Cai
 		5f77e1292d Support Multi Process Tokenizer Manager(#6555) (#8964) 
Signed-off-by: ybyang <ybyang7@iflytek.com>
Signed-off-by: huanglong <huanglong@linux.alibaba.com>
Co-authored-by: Huang Long <121648372+LLLL114@users.noreply.github.com>
Co-authored-by: huanglong <huanglong@linux.alibaba.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-09-01 01:00:13 -07:00
Teng Maandhzh0425 f05c68733e [HiCache] Clear kvcache in storage backend with fastAPI (#9750) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
 2025-08-31 17:41:44 +08:00
Chanh NguyenChanh NguyenLiangsheng Yin
 		127d4b0d5e Support GC Freezing to improve latency & throughput (#9241) 
Co-authored-by: Chanh Nguyen <cnguyen@linkedin.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
 2025-08-23 13:43:09 +08:00
Chengxing Xie c1c7dc4534 feat: Add model version tracking with API endpoints and response metadata (#8795) 2025-08-14 12:13:46 -07:00
Chang Su a218490136 (gpt-oss, oai, chat): Remove Harmony Integration and Implement Native GPT-OSS Tool Call Support (#9043) 2025-08-11 18:59:18 -07:00
Lianmin Zheng 4ea9d74a3e Simplify health check (#9034) 2025-08-10 17:35:05 -07:00
Zilin Zhu dd650e0e21 [RL] fix skip_server_warmup and rl health_generate logic (#8757) 2025-08-08 04:34:38 -07:00
Lianmin Zheng a947154286 Revert "Support Multi Process Tokenizer Manager" (#8960) 2025-08-08 02:28:27 -07:00
ybyanglw9527huanglongHuang Long
 		7490e3f67d Support Multi Process Tokenizer Manager (#6555) 
Signed-off-by: ybyang <ybyang7@iflytek.com>
Signed-off-by: huanglong <huanglong@linux.alibaba.com>
Co-authored-by: lw9527 <952799980@qq.com>
Co-authored-by: huanglong <huanglong@linux.alibaba.com>
Co-authored-by: Huang Long <121648372+LLLL114@users.noreply.github.com>
 2025-08-08 01:45:50 -07:00
Chang SuXinyuan TongXinyuan Tong
 		92cc32d9fc Support v1/responses and use harmony in serving_chat (#8837) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-08-06 16:20:34 -07:00
ybyangLianmin ZhengKan Wu
 		6f9baf1002 [Improvements] Merge health check route (#8444) 
Signed-off-by: ybyang <ybyang7@iflytek.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Kan Wu <wukanustc@gmail.com>
 2025-08-03 01:59:06 -07:00
Lianmin Zheng a4c3b121d8 Split the scheduler into multiple mixin classes to reduce the file size (#8483) 2025-07-29 12:46:50 -07:00
harrisonlimh 747dd45077 feat: throttle requests at scheduler based on --max_queued_requests (#7565) 2025-07-28 22:32:33 +08:00
Lianmin Zheng ed2e313eb6 Clean up server_args, triton cache manager (#8332) 2025-07-25 14:14:51 -07:00
Lianmin Zheng 55381a46ac Revert "[Feature] Simple Improve Health Check Mechanism for Production-Grade Stability" (#8181) 2025-07-19 22:41:30 -07:00
ybyang 4540a4666a [Feature] Simple Improve Health Check Mechanism for Production-Grade Stability (#8115) 
Signed-off-by: ybyang <ybyang7@iflytek.com>
 2025-07-19 18:10:00 -07:00
kyleliang-nv dd445a41f5 [feature] Add start step profile argument in /start_profile (#7608) 2025-07-09 18:42:15 -07:00
Zilin Zhu af46f299f9 [RL] add pause and continue generation for async rl training (#7419) 2025-07-04 18:49:49 -07:00
AlbertandZilin Zhu d3c275b117 Support updating weights at once by stopping all requests (#6698) 
Signed-off-by: Tianyu Zhou <albert.zty@antgroup.com>
Co-authored-by: Zilin Zhu <zhuzilinallen@gmail.com>
 2025-07-02 22:26:06 -07:00
Zilin Zhu 09e699bba4 [RL] add --skip-warmup (#7416) 2025-07-02 18:50:43 -07:00
Lianmin ZhengandKan Wu 22352d47a9 Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632) 
Co-authored-by: Kan Wu <wukanustc@gmail.com>
 2025-06-29 23:16:19 -07:00
Lianmin Zheng 071a1f51ae [Minor] clean up multimodal processor and tokenizer manager (#7624) 2025-06-29 02:50:14 -07:00
Lifu Huang 49538d111b Support dynamic LoRA loading / unloading in engine/server API (#7446) 2025-06-27 21:00:27 -07:00
Chang Su d04163b3fa Fix RequestValidationError response format (#7487) 2025-06-23 20:35:11 -07:00
Chang SuandJin Pan 72676cd6c0 feat(oai refactor): Replace openai_api with entrypoints/openai (#7351) 
Co-authored-by: Jin Pan <jpan236@wisc.edu>
 2025-06-21 13:21:06 -07:00
woodx e30ef368ab Feat/support rerank (#6058) 2025-06-16 10:50:01 -07:00
Byron Hsu 7d316991b2 [PD] Update prefill.py (#7190) 2025-06-14 15:59:54 -07:00
Chanh NguyenandChanh Nguyen 3f1e433903 Decoder-only Scoring API (#6460) 
Co-authored-by: Chanh Nguyen <cnguyen@linkedin.com>
 2025-06-04 14:14:54 -07:00
Lianmin Zheng 20fd53b8f6 Correctly abort the failed grammar requests & Improve the handling of abort (#6803) 2025-06-01 19:00:07 -07:00
Lianmin Zheng 2d72fc47cf Improve profiler and integrate profiler in bench_one_batch_server (#6787) 2025-05-31 15:53:55 -07:00
Liangsheng Yinandfzyzcjy 78689d3393 PD Rust LB (PO2) (#6437) 
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
 2025-05-29 20:50:10 +08:00
Chang Su ed0c3035cd feat(Tool Calling): Support required and specific function mode (#6550) 2025-05-23 21:00:37 -07:00
fzyzcjy 01d2838c0f Fix stop_profile does not wait for finishing (#4741) 2025-05-17 17:06:15 -07:00
Lifu Huang 3cf1473a09 Use monotonic clock for interval measurement (#6211) 
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
 2025-05-17 16:49:18 -07:00
Lianmin Zheng e07a6977e7 Minor improvements of TokenizerManager / health check (#6327) 2025-05-15 15:29:25 -07:00
Lianmin ZhengandSangBin Cho fba8eccd7e Log if cuda graph is used & extend cuda graph capture to cuda-graph-max-bs (#6201) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
 2025-05-12 00:17:33 -07:00
fzyzcjy 3f2702ae51 Fix start_profile does not support with_stack and record_shapes (#6043) 2025-05-11 23:11:32 -07:00
Lianmin Zheng de167cf5fa Fix request abortion (#6184) 2025-05-10 21:54:46 -07:00
fzyzcjy cef91b1ed7 [PD] Add control to slow down a server (#5572) 2025-05-08 01:03:08 -07:00