chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
10,225 Commits 1 Branch 0 Tags
d939e26585ac8b319793895823daddd28863d6b1
Commit Graph

105 Commits

Author SHA1 Message Date
Liangsheng Yin
e6d40bff81 Revert "feat: reduce constrained-decoding overhead in TP" (#16845) 2026-01-10 11:39:38 +08:00
Raayan Dhar
76b3c698d6 feat: reduce constrained-decoding overhead in TP (#13947) 
Signed-off-by: Raayan Dhar raayan.dhar@gmail.com <raayan.dhar@gmail.com>
Signed-off-by: raayandhar <raayan.dhar@gmail.com>
 2026-01-10 00:38:32 +08:00
Qiaolin Yu
dd93e4451c Support returning customized info (#16144) 2026-01-02 15:32:18 +08:00
Lianmin Zheng
e4c1e441af Fix deprecation warning of sre_parse for python 3.13 (#16247) 2025-12-31 21:25:09 -08:00
Qiaolin Yu
173940927f Add customized sampler registration (#15423) 2025-12-18 23:10:23 -08:00
Binyao Jiang
cf0478d602 [Glm46v] Bug fix for accuracy drop and unable to launch server (#14585) 
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
 2025-12-07 23:45:02 -08:00
Lianmin Zheng
bc3d2a85af [Minor] update docs (#14212) 2025-12-01 02:33:58 -08:00
0xNullPath
70562969b9 [Bug] OOM (Out-of-Memory) errors for extreme testing scenarios (min_tokens=2) (#11757) 
Signed-off-by: Yan Lu <luyan@nvidia.com>
 2025-11-01 00:28:41 +08:00
Xinyuan Tong
684864814b Feat: deepseek-ocr logits processor (#12415) 
Co-authored-by: xinyuant <xinyuant@usc.edu>
 2025-10-31 23:35:22 +08:00
ybyang
dbb16bedd5 Support Thinking Budget (via custom_logit_processor for OpenAI API) [Fix #6572] (#11416) 
Signed-off-by: ybyang <ybyang7@iflytek.com>
Co-authored-by: YorkSu <york_su@qq.com>
 2025-10-21 16:27:56 +08:00
Lianmin Zheng
5e3f7e7fa9 Minor: improve sampler & remove unused fields from model_config.py (#11531) 2025-10-13 11:04:44 -07:00
Liangsheng Yin
516738b096 Depreate global_server_args_dict (#11528) 2025-10-13 19:34:43 +08:00
Cheng Wan
1bdd010291 Revert "Deprecate global_server_args_dict" (#11520) 2025-10-12 17:40:40 -07:00
Liangsheng Yin
1083e7e3df Deprecate global_server_args_dict (#11331) 2025-10-13 01:20:47 +08:00
Vincent Zhong
a220536f40 [ perf ] Replace json-> orjson in hot path (#11221) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2025-10-12 20:30:58 +08:00
Glen Liu
47c606d3dc [Feature] support regex strings as a stopping condition (#10635) 2025-10-12 10:53:15 +08:00
Liangsheng Yin
501dfa6b42 Remove sampling info events and overlap thread file (#11300) 2025-10-07 21:34:25 +08:00
Liangsheng Yin
1519a89cfd Remove overlap thread (#11210) 
Co-authored-by: Lianmin Zheng <15100009+merrymercy@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
 2025-10-07 20:12:12 +08:00
Alex Chi Z
d01b921482 fix sampling_seed handling when deterministic is enabled (#11096) 
Signed-off-by: Alex Chi <iskyzh@gmail.com>
 2025-10-03 20:41:46 -07:00
Lianmin Zheng
dda34c2f93 Fix mem fraction static for nightly tests (#11076) 2025-09-29 12:57:41 -07:00
Qiaolin Yu
e2ac7888b8 [2/2] Support deterministic inference for temperature > 0 (#10678) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
 2025-09-21 19:36:08 -07:00
Baizhou Zhang
8ecef73f12 [1/2] Support deterministic inference with flashinfer attention backend (#10645) 
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
 2025-09-19 23:34:29 -07:00
Lianmin Zheng
71133a0426 [Auto Sync] Update sampling_batch_info.py (20250909) (#10212) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: cctry <shiyang@x.ai>
 2025-09-09 01:29:52 -07:00
yhyang201
c377923304 [feat] Reduce GPU memory overhead by using weakref (#9673) 2025-08-28 01:09:06 -07:00
Lianmin Zheng
9e426466af Clean up allocators (#9134) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-08-13 13:56:04 -07:00
J
0e7a5b2694 fix: prevent crashes due to logit bias dimension mismatch (#7685) 2025-07-23 15:30:55 -07:00
ehuaa
0c55cbcfc5 [BugFix] add verify logit_bias to avoid crash because of IndexError (#7749) 2025-07-14 02:44:12 +08:00
Lianmin Zheng
071a1f51ae [Minor] clean up multimodal processor and tokenizer manager (#7624) 2025-06-29 02:50:14 -07:00
Brayden Zhong
ca9291181d [Feature] Add Logit Bias (#6579) 
Co-authored-by: Cinjon Resnick <cinjon.resnick@gmail.com>
 2025-06-10 15:39:25 -07:00
Lianmin Zheng
608668e143 Slightly improve the sampler to skip unnecessary steps (#6956) 2025-06-08 03:18:54 -07:00
Lianmin Zheng
ac2324c177 Skip the flaky test_stateful_custom_logit_processor (#6251) 2025-05-12 18:29:41 -07:00
Lianmin Zheng
d18c6b3358 Support incremental streaming of logprob/token_ids between scheduler and detokenizer (#6225) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
 2025-05-12 14:33:38 -07:00
applesaucethebun
2ce8793519 Add typo checker in pre-commit (#6179) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-05-11 12:55:00 +08:00
Lianmin Zheng
de167cf5fa Fix request abortion (#6184) 2025-05-10 21:54:46 -07:00
Yineng Zhang
66fc63d6b1 Revert "feat: add thinking_budget (#6089)" (#6181) 2025-05-10 16:07:45 -07:00
thyecust
63484f9fd6 feat: add thinking_budget (#6089) 2025-05-09 08:22:09 -07:00
Lianmin Zheng
177320a582 Clean up imports (#5467) 2025-04-16 15:26:49 -07:00
JieXin Liang
1a3fa75f2f [Fix] use torch.cat instead of torch.concat to prevent entering the Autograd backends. (#4466) 2025-03-16 00:02:47 -07:00
Wang Ran (汪然)
2892b9bb97 bugfix: Update sampling_params.py (#4413) 2025-03-15 16:39:19 -07:00
Lianmin Zheng
d4017a6b63 [EAGLE] many fixes for eagle (#4195) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>
 2025-03-07 22:12:13 -08:00
Qiaolin Yu
57a404fd55 Remove outdated test utils and fix links for the doc of sampling params (#3999) 2025-03-03 09:41:38 -08:00
Lianmin Zheng
935cda944b Misc clean up; Remove the support of jump forward (#4032) 2025-03-03 07:02:14 -08:00
Lianmin Zheng
ac2387279e Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>
 2025-03-03 00:12:04 -08:00
Qiaolin Yu
40782f05d7 Refactor: Move return_hidden_states to the generate input (#3985) 
Co-authored-by: Beichen-Ma <mabeichen12@gmail.com>
 2025-03-01 17:51:29 -08:00
mlmz
bac414ab53 [Feature] integrate Structural Tag in xgrammar backend for function calling (#3566) 
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
 2025-02-27 23:33:41 -08:00
Qiaolin Yu
d38878523d Fix the doc link for sampling params (#3861) 2025-02-27 13:31:43 -08:00
Qiaolin Yu
d6898dd253 Add return hidden state in the native API (#3897) 
Co-authored-by: Beichen-Ma <mabeichen12@gmail.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
 2025-02-26 22:06:54 -08:00
Lianmin Zheng
4f118a39d7 Fix repetition penalty (#3139) 2025-01-25 21:48:58 -08:00
Lianmin Zheng
27acf63bbd Use torch.compile for scaling penalty (#3133) 2025-01-25 18:27:33 -08:00
Hongpeng Guo
583697cd71 [Enhancement] Custom Logit Processor Improvement (#2998) 
Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
 2025-01-20 02:00:35 -08:00