chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
8,931 Commits 1 Branch 0 Tags
e7dc85c50baf434fcd5faeee30a4e1d0cd9ddaed
Commit Graph

147 Commits

Author SHA1 Message Date
Yi Zhong
d1110e1c3e docs only add kimi k2 thinking and kimi linear (#15789) 2026-01-15 12:09:52 -05:00
shuwenn
9227d9f60c [Docs] sort and update server_arguments.md (#17163) 2026-01-15 12:07:18 -05:00
Glen Liu
6b065298b5 [Docs] add routing-key to schedule-policy in docs (#17101) 2026-01-14 22:22:07 -05:00
fxmarty-amd
5af84c8af5 [AMD][Quantization] Add int4fp8_moe online quantization on ROCm (#7392) 
Co-authored-by: Dehua Tang <dehtang@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: YC Tseng <yctseng@amd.com>
 2026-01-14 01:44:40 -08:00
shuwenn
cd33694585 feat: add --admin-api-key for finer-grained endpoint auth (#15908) 
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
 2026-01-13 20:21:55 -08:00
James
ae0baefb94 [NPU] upgrade npu mf_apater plugin (#15853) 2026-01-13 09:02:10 +08:00
Wenyi Xu
3c16c58619 [model-gateway] Add Redis support as a history backend (#16300) 2026-01-11 01:03:00 -08:00
Ratish P
c0248d6f37 [dpc]: unify DP controller load balancing and simplify dispatch logic (#16258) 2026-01-11 12:38:03 +08:00
Shangming Cai
973116e6bb [Doc] Optimize pipeline parallelism doc (#16630) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-01-07 14:52:42 +08:00
Huapeng Zhou
078270473a [Doc] Default lora backend: csgmv (#16444) 2026-01-05 12:45:49 +08:00
Yuan Luo
9bd64d739b [VLM] Add doc for ViT CUDA Graph (#16343) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-04 10:05:23 +08:00
Yongfei Xu
0d244116d2 [DeepSeek v3.2] opt Context Parallelism: support fused moe, multi batch and fp8 kvcache (#13959) 2026-01-02 23:49:14 +08:00
Huaixin Chang
c1dfbc777b deprecate prefill-round-robin-balance (#16195) 
Signed-off-by: Chang Huaixin (OpenAnolis) <changhuaixin@linux.alibaba.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-12-31 22:25:33 +08:00
Simo Lin
4bc2f2e0bc [model-gateway][docs] Add Classification API documentation (#16182) 
Co-authored-by: Chang Su <chang.s.su@oracle.com>
 2025-12-30 10:53:31 -08:00
Mufeez Amjad
cbff7ad985 dp-attention: add follow_bootstrap_room + auto load-balance; drop decode_round_robin (#16110) 2025-12-30 22:33:06 +08:00
Simo Lin
de2799f3f5 [docs] Fix non-clickable ToC links in model gateway documentation (#16054) 2025-12-28 23:35:37 -08:00
Baizhou Zhang
208e6a9dac [Doc]Update MTP moe backends for EP document (#16013) 2025-12-28 19:36:23 +08:00
Ho-Ren (Jack) Chuang
349ce2dd19 Support kv8 (FP8) with torch_native attention backend (#12596) 
Signed-off-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
 2025-12-28 10:48:30 +08:00
Simo Lin
9665574937 [docs] major SGL Model Gateway documentation update (#15715) 2025-12-23 20:26:09 -08:00
Shangming Cai
b3f83cc1c5 Fix pipeline parallelism doc typos (#15688) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-12-23 22:19:15 +08:00
Shangming Cai
c15fa1c577 [2/N] Update doc of Pipeline Parallelism with case study (#15684) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-12-23 20:56:15 +08:00
shuwenn
1d9ba2ce4d [model-gateway] Use UUIDs for router-managed worker resources (#15540) 2025-12-21 07:40:03 -10:00
Xinyuan Tong
0a346d3bd9 feat: Add limit-mm-data-per-request argument to server arguments (#15418) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-20 10:44:47 -08:00
mlmz
1f1f05a85e vlm: refactor engine vlm params and support processor output as input (#14091) 
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: zhaochenyang20 <zhaochenyang20@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: BenYao21 <cyao22@asu.edu>
Co-authored-by: minleminzui <minleminzui@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
 2025-12-20 18:31:24 +08:00
Yuxuan Zhang
b82c7a0ae7 [GLM-4.7] GLM-4.7 Tool Parser and Doc Update (#15333) 2025-12-19 20:30:44 -08:00
elvischenv
435d1c83c1 [Perf] Enable Flashinfer autotune by default (#14357) 2025-12-16 23:01:39 -08:00
Baizhou Zhang
d92c1f8cbd Fix lora doc (#15282) 2025-12-16 15:53:38 -08:00
amysaq2023
ccc8f3b266 support non disturbing remote instance weight loader v2 (#14997) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-16 14:39:56 -08:00
Tianyu Guo
a9a2cdd8ec Add EPD disaggregation doc (#15224) 2025-12-16 11:22:05 +08:00
Hanming Lu
e61dabf5e4 [Qwen3-next] support mamba radix cache for overlap scheduler (#14792) 2025-12-14 18:54:16 -08:00
Ho-Ren (Jack) Chuang
171b442ad3 Add KV4-capable backend flashmla and update server args (#14989) 
Signed-off-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
 2025-12-12 11:50:27 -08:00
Yineng Zhang
4b7b5af36a Revert several PRs (#14958) 
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-12-12 11:25:12 -08:00
Shangming Cai
c8cf1cafdb [1/N] Update doc of Pipeline Parallelism (#14985) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: alpha-baby <fujianhao1997@qq.com>
 2025-12-12 19:32:52 +08:00
Ho-Ren (Jack) Chuang
10146af099 Check KV4 compatibility with attention backends and add KV4 support to the attention_backend doc (#14467) 
Signed-off-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
 2025-12-11 19:00:53 -08:00
Xinyuan Tong
9975acf50f [refactor] Update reasoning parameter to require_reasoning (#14922) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 16:55:59 -08:00
amysaq2023
70758d457e support non-disturbing remote-instance-weight-loader (#13125) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-11 16:45:32 -08:00
fzyzcjy
d85fecb536 Tiny fix incorrect worker removal command (#14822) 2025-12-10 22:29:39 +08:00
b8zhong
55504df2f7 Add FP8 Blockwise GEMM Backend Flag --fp8-gemm-backend (#14379) 2025-12-09 12:05:56 -08:00
Muqi Li
06836ad02a [Reasoning + Structured Output] make reasoning compatible with structured output (#12551) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-08 01:28:39 -08:00
Binyao Jiang
cf0478d602 [Glm46v] Bug fix for accuracy drop and unable to launch server (#14585) 
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
 2025-12-07 23:45:02 -08:00
fzyzcjy
1915a1f8a7 Super tiny remove unneeded policy flag (#14608) 2025-12-07 19:35:07 -08:00
Zhiyu
f6423b626c Rename TensorRT Model Optimizer to Model Optimizer (#14455) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
 2025-12-07 13:18:20 -08:00
b8zhong
dd91d38e6a [Doc] Add short explanation on page size (#14557) 2025-12-06 17:26:30 -08:00
Feng Su
e12c6b320f [model-gateway][tracing]: implement request tracing using OpenTelemetry with trace context propagation (HTTP) (#13897) 2025-12-06 05:59:04 -08:00
Rain Jiang
ea177372bd support mtp with deepseek r1 nvfp4 model (#13115) 
Co-authored-by: Trevor Morris <tmorris@nvidia.com>
 2025-12-06 00:45:54 -08:00
Baizhou Zhang
42fcf5438f Revert "tiny remove deprecated endpoint call" (#14533) 2025-12-05 23:48:54 -08:00
Yuhao Yang
7b0c7ad163 Revise DP Multi-Modal Encoder Document (#14290) 2025-12-06 12:56:53 +08:00
b8zhong
ec7b2c16d9 tiny remove deprecated endpoint call (#13607) 2025-12-05 09:54:49 -08:00
b8zhong
88d1bab537 add doc for quantized kv cache (#14348) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
Co-authored-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
 2025-12-04 13:01:05 -08:00
b8zhong
65c8568c4a sync attention, deepseek doc (#14335) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-02 21:19:40 -08:00