chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
6,407 Commits 1 Branch 0 Tags
c0652d907b2ee39d010598ec4c5f7f287fa41947
Commit Graph

75 Commits

Author SHA1 Message Date
Lianmin Zheng
c0652d907b Clean up sgl kernel (#12413) 
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
 2025-10-31 01:13:34 -07:00
Xiaoyu Zhang
334543ff3b Add continuous_usage_stats support for streaming responses (#12241) 2025-10-29 10:01:23 +08:00
Chenxi Li
cc7b04a29c Feature/Add GET endpoint to query loaded LoRA adapters (#12229) 2025-10-28 01:22:00 -07:00
satyamk7054
9fc3e8aac7 Add support for Matryoshka embeddings (#126) (#11142) 
Co-authored-by: Satyam Kumar <satyamk@linkedin.com>
 2025-10-28 02:49:36 +08:00
Liangsheng Yin
9d61205dac [lint] improve ruff check (#11922) 
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2025-10-22 11:32:50 +08:00
Chang Su
70f6309cd4 [router][grpc] Support v1/responses API (#11926) 2025-10-21 17:41:48 -07:00
ybyang
dbb16bedd5 Support Thinking Budget (via custom_logit_processor for OpenAI API) [Fix #6572] (#11416) 
Signed-off-by: ybyang <ybyang7@iflytek.com>
Co-authored-by: YorkSu <york_su@qq.com>
 2025-10-21 16:27:56 +08:00
Neelabh Sinha
852c0578fd [FEATURE] Add OpenAI-Compatible LoRA Adapter Selection (#11570) 2025-10-21 15:44:33 +08:00
DarkSharpness
276e7b3e4e [Feature] New structural tag support (#10691) 2025-10-20 18:25:58 +08:00
ybyang
b5e14b2b78 [1/2][feature] support openai like classification api (#11618) 2025-10-18 19:32:48 -07:00
Chang Su
f226d3da2a Fix missing json imports in serving_responses.py (#11681) 2025-10-15 13:01:55 -07:00
Vincent Zhong
a220536f40 [ perf ] Replace json-> orjson in hot path (#11221) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2025-10-12 20:30:58 +08:00
Glen Liu
47c606d3dc [Feature] support regex strings as a stopping condition (#10635) 2025-10-12 10:53:15 +08:00
Adarsh Shirawalmath
7c3f07dbcb [Feature] Add /tokenize and /detokenize OpenAI compatible endpoints (#9545) 2025-10-08 12:38:48 +08:00
Chang Su
7ba3de0e92 [oai serving chat] Add argument --sampling-defaults and fix ChatCompletionRequest defaults (#11304) 2025-10-08 00:36:05 +00:00
Vincent Zhong
36a6b8dbfc Update v1/responses to be more OpenAI-compatible. (#9624) 2025-10-05 18:47:46 +00:00
Jimmy
3713eb6135 feat(reasoning): improve enable thinking from request (#10875) 2025-09-29 16:59:08 -07:00
Tejesh Anand
8cc27fdc46 Use jsonschema to constrain required or specific tool choice (#10550) 2025-09-27 13:18:50 -04:00
eraser00
0ac6114694 Replace the Kimi-K2 generated tool call idx with history tool call count (#10612) 
Co-authored-by: eraser00 <eraser00@github.com>
 2025-09-25 18:47:40 -07:00
Lianmin Zheng
f68dd998b9 Rename customer label -> custom label (#10899) 
Co-authored-by: Yingchun Lai <laiyingchun@apache.org>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-25 16:19:53 -07:00
Xinyuan Tong
71f24ef8f6 feat: add cache_salt support to request (#10718) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-09-23 23:30:25 -07:00
Yineng Zhang
42aedb02af [Auto Sync] Update protocol.py (20250923) (#10820) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: cicirori <32845984+cicirori@users.noreply.github.com>
 2025-09-23 12:49:56 -07:00
harrisonlimh
14fdd52740 feat: add priority based scheduling with priority based request acceptance and preemption (#8746) 2025-09-16 17:10:10 -07:00
Vincent Zhong
0b14159fc4 Add reasoning examples for GPT-OSS in Markdown examples (#9626) 2025-09-15 11:27:40 +08:00
Yingchun Lai
fc2c3a3d8e metrics: support customer labels specified in request header (#10143) 2025-09-14 20:00:08 -07:00
Lianmin Zheng
033b75f559 [Auto Sync] Update serving_base.py, serving_chat.py, servin... (20250910) (#10282) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: cctry <shiyang@x.ai>
 2025-09-10 16:58:59 -07:00
Xinyuan Tong
f3b5db6ee8 Feat: support disable tool parser (#10184) 2025-09-10 14:03:55 -07:00
Tony Lu
1e18a341e9 [Bugfix] fix pd chat completion protocol for batching support (#10016) 
Signed-off-by: Tony Lu <tonylu@linux.alibaba.com>
 2025-09-04 01:43:16 -07:00
Lianmin Zheng
60e37f8028 Move parsers under a single folder (#9912) 2025-09-02 18:25:04 -07:00
gongwei-130
3fd1431df2 support enable in the reasoning field to enable thingking for thinkin… (#9715) 2025-08-29 10:57:32 -07:00
gongwei-130
9a7c8842ba accomendate json schema in the "schema" field, not in "json_schema" field of response_format (#9786) 2025-08-28 23:51:50 -07:00
Xinyuan Tong
68a54e063e Sets default model name in request classes (#9683) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-08-27 10:43:03 -07:00
cicirori
b6c14ec0b4 add response_format support for completion API (#9665) 2025-08-26 15:01:29 -07:00
Xiaotong Jiang
0936c766ed Fix kimi k2 function calling format (#9606) 2025-08-26 00:50:59 -07:00
GavinZhu-GMI
0ef583b7de fix: allow user to specify function as role (#9635) 2025-08-26 00:47:20 -07:00
Jonas
a0a77d937b Fix Harmony reasoning parser for and auto-separation for gpt-oss models (#9190) 
Co-authored-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: zhaochenyang20 <zhaochenyang20@gmail.com>
Co-authored-by: minleminzui <2969413251@qq.com>
Co-authored-by: maocheng23 <maocheng@berkeley.edu>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-08-25 15:26:26 -07:00
Binyao Jiang
3affa9dcc3 Fix GLM45 tool call multi-turn bug (#9500) 2025-08-25 13:46:13 -07:00
Xinyuan Tong
e8449ab515 Add deepseek v3.1 thinking parser support and update docs (#9464) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-08-21 15:09:40 -07:00
gongwei-130
10d34f74e2 fix: should return a invalid request response when schema missing (#9461) 2025-08-21 14:06:50 -07:00
gongwei-130
9ba7253094 accomendate reasoning_effort set in chat_template_kwargs (#9458) 2025-08-21 13:22:03 -07:00
hlu1
dae9a80f43 [fix] Fix mxfp4 weight loading bug with TP sharding in GPT-OSS (#9433) 
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-08-21 03:50:51 -07:00
Keyang Ru
f515449582 Fix gpt-oss response api streaming issue (#9368) 2025-08-19 20:19:42 -07:00
gongwei-130
0cf3fbeb18 should return invalide request for empty prompt (#9315) 2025-08-18 11:44:11 -07:00
Chengxing Xie
c1c7dc4534 feat: Add model version tracking with API endpoints and response metadata (#8795) 2025-08-14 12:13:46 -07:00
Chang Su
f2a5de284b [Bugfix] Fix accuracy-test-1-gpu failure caused by builtin_tools (#9114) 2025-08-12 09:56:13 -07:00
Chang Su
a218490136 (gpt-oss, oai, chat): Remove Harmony Integration and Implement Native GPT-OSS Tool Call Support (#9043) 2025-08-11 18:59:18 -07:00
Lianmin Zheng
9a44b643c6 Fix CI (#9012) 2025-08-09 13:33:42 -07:00
Xinyuan Tong
3e7ff1ab1f fix: reasoning parser when request have enable_thinking flag (#8933) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-08-07 15:52:06 -07:00
Xinyuan Tong
3fa3c6cd6a Enables force reasoning based on chat template for Qwen3-Thinking (#8369) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Chang Su <csu272@usc.edu>
 2025-08-06 20:02:47 -07:00
Chang Su
92cc32d9fc Support v1/responses and use harmony in serving_chat (#8837) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-08-06 16:20:34 -07:00