chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
9,135 Commits 1 Branch 0 Tags
7ace64d1d88de1dfd3cfb6622f3fcbba4fe7eef8
Commit Graph

156 Commits

Author SHA1 Message Date
hxie
13f88045b3 configuration file support and nixl integration augmentation for hicache-storage-backend-extra-config (#16602) 2026-01-22 14:31:48 -08:00
zijiexia
4ecd9afde9 [Docs] Rename SGLang Router to SGLang Model Gateway (#17436) 2026-01-20 12:31:10 -08:00
Ruihuan He
eb38d64413 [Docs] Explain CUDA attention backend choices and aiter FP8 KV cache support (#17428) 2026-01-20 11:10:36 -08:00
Shangming Cai
23d765d1f2 [Doc] Update pipeline parallelism documentation (#17414) 2026-01-20 20:10:56 +08:00
b8zhong
f374623fa9 [Refactor] Set fp4-gemm-backend=auto on SM100 and rename fp4-gemm-backend with flashinfer_ prefix (#17309) 2026-01-19 20:09:07 +08:00
Glen Liu
ad1b4e4728 [Feature] overlap LoRA weight loading with compute (#15512) 2026-01-19 10:43:17 +08:00
Xinyuan Tong
2069050d3f fix: Handle multiple named chat templates in HuggingFace tokenizers (#17236) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-01-18 17:20:04 +08:00
zijiexia
166396ca4c [Docs] minor update on ep docs (#17242) 2026-01-16 18:20:47 -08:00
shuwenn
8ec160ed46 feature: support uvicorn access log filter(disable logging /metrics) (#15513) 2026-01-15 20:00:06 -08:00
Yi Zhong
d1110e1c3e docs only add kimi k2 thinking and kimi linear (#15789) 2026-01-15 12:09:52 -05:00
shuwenn
9227d9f60c [Docs] sort and update server_arguments.md (#17163) 2026-01-15 12:07:18 -05:00
Glen Liu
6b065298b5 [Docs] add routing-key to schedule-policy in docs (#17101) 2026-01-14 22:22:07 -05:00
fxmarty-amd
5af84c8af5 [AMD][Quantization] Add int4fp8_moe online quantization on ROCm (#7392) 
Co-authored-by: Dehua Tang <dehtang@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: YC Tseng <yctseng@amd.com>
 2026-01-14 01:44:40 -08:00
shuwenn
cd33694585 feat: add --admin-api-key for finer-grained endpoint auth (#15908) 
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
 2026-01-13 20:21:55 -08:00
James
ae0baefb94 [NPU] upgrade npu mf_apater plugin (#15853) 2026-01-13 09:02:10 +08:00
Wenyi Xu
3c16c58619 [model-gateway] Add Redis support as a history backend (#16300) 2026-01-11 01:03:00 -08:00
Ratish P
c0248d6f37 [dpc]: unify DP controller load balancing and simplify dispatch logic (#16258) 2026-01-11 12:38:03 +08:00
Shangming Cai
973116e6bb [Doc] Optimize pipeline parallelism doc (#16630) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-01-07 14:52:42 +08:00
Huapeng Zhou
078270473a [Doc] Default lora backend: csgmv (#16444) 2026-01-05 12:45:49 +08:00
Yuan Luo
9bd64d739b [VLM] Add doc for ViT CUDA Graph (#16343) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-04 10:05:23 +08:00
Yongfei Xu
0d244116d2 [DeepSeek v3.2] opt Context Parallelism: support fused moe, multi batch and fp8 kvcache (#13959) 2026-01-02 23:49:14 +08:00
Huaixin Chang
c1dfbc777b deprecate prefill-round-robin-balance (#16195) 
Signed-off-by: Chang Huaixin (OpenAnolis) <changhuaixin@linux.alibaba.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-12-31 22:25:33 +08:00
Simo Lin
4bc2f2e0bc [model-gateway][docs] Add Classification API documentation (#16182) 
Co-authored-by: Chang Su <chang.s.su@oracle.com>
 2025-12-30 10:53:31 -08:00
Mufeez Amjad
cbff7ad985 dp-attention: add follow_bootstrap_room + auto load-balance; drop decode_round_robin (#16110) 2025-12-30 22:33:06 +08:00
Simo Lin
de2799f3f5 [docs] Fix non-clickable ToC links in model gateway documentation (#16054) 2025-12-28 23:35:37 -08:00
Baizhou Zhang
208e6a9dac [Doc]Update MTP moe backends for EP document (#16013) 2025-12-28 19:36:23 +08:00
Ho-Ren (Jack) Chuang
349ce2dd19 Support kv8 (FP8) with torch_native attention backend (#12596) 
Signed-off-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
 2025-12-28 10:48:30 +08:00
Simo Lin
9665574937 [docs] major SGL Model Gateway documentation update (#15715) 2025-12-23 20:26:09 -08:00
Shangming Cai
b3f83cc1c5 Fix pipeline parallelism doc typos (#15688) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-12-23 22:19:15 +08:00
Shangming Cai
c15fa1c577 [2/N] Update doc of Pipeline Parallelism with case study (#15684) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-12-23 20:56:15 +08:00
shuwenn
1d9ba2ce4d [model-gateway] Use UUIDs for router-managed worker resources (#15540) 2025-12-21 07:40:03 -10:00
Xinyuan Tong
0a346d3bd9 feat: Add limit-mm-data-per-request argument to server arguments (#15418) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-20 10:44:47 -08:00
mlmz
1f1f05a85e vlm: refactor engine vlm params and support processor output as input (#14091) 
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: zhaochenyang20 <zhaochenyang20@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: BenYao21 <cyao22@asu.edu>
Co-authored-by: minleminzui <minleminzui@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
 2025-12-20 18:31:24 +08:00
Yuxuan Zhang
b82c7a0ae7 [GLM-4.7] GLM-4.7 Tool Parser and Doc Update (#15333) 2025-12-19 20:30:44 -08:00
elvischenv
435d1c83c1 [Perf] Enable Flashinfer autotune by default (#14357) 2025-12-16 23:01:39 -08:00
Baizhou Zhang
d92c1f8cbd Fix lora doc (#15282) 2025-12-16 15:53:38 -08:00
amysaq2023
ccc8f3b266 support non disturbing remote instance weight loader v2 (#14997) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-16 14:39:56 -08:00
Tianyu Guo
a9a2cdd8ec Add EPD disaggregation doc (#15224) 2025-12-16 11:22:05 +08:00
Hanming Lu
e61dabf5e4 [Qwen3-next] support mamba radix cache for overlap scheduler (#14792) 2025-12-14 18:54:16 -08:00
Ho-Ren (Jack) Chuang
171b442ad3 Add KV4-capable backend flashmla and update server args (#14989) 
Signed-off-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
 2025-12-12 11:50:27 -08:00
Yineng Zhang
4b7b5af36a Revert several PRs (#14958) 
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-12-12 11:25:12 -08:00
Shangming Cai
c8cf1cafdb [1/N] Update doc of Pipeline Parallelism (#14985) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: alpha-baby <fujianhao1997@qq.com>
 2025-12-12 19:32:52 +08:00
Ho-Ren (Jack) Chuang
10146af099 Check KV4 compatibility with attention backends and add KV4 support to the attention_backend doc (#14467) 
Signed-off-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
 2025-12-11 19:00:53 -08:00
Xinyuan Tong
9975acf50f [refactor] Update reasoning parameter to require_reasoning (#14922) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 16:55:59 -08:00
amysaq2023
70758d457e support non-disturbing remote-instance-weight-loader (#13125) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-11 16:45:32 -08:00
fzyzcjy
d85fecb536 Tiny fix incorrect worker removal command (#14822) 2025-12-10 22:29:39 +08:00
b8zhong
55504df2f7 Add FP8 Blockwise GEMM Backend Flag --fp8-gemm-backend (#14379) 2025-12-09 12:05:56 -08:00
Muqi Li
06836ad02a [Reasoning + Structured Output] make reasoning compatible with structured output (#12551) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-08 01:28:39 -08:00
Binyao Jiang
cf0478d602 [Glm46v] Bug fix for accuracy drop and unable to launch server (#14585) 
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
 2025-12-07 23:45:02 -08:00
fzyzcjy
1915a1f8a7 Super tiny remove unneeded policy flag (#14608) 2025-12-07 19:35:07 -08:00