chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
8,020 Commits 1 Branch 0 Tags
af780c59f3a4e911dfbbbce7652ea8bbbf0063fd
Commit Graph

122 Commits

Author SHA1 Message Date
elvischenv
435d1c83c1 [Perf] Enable Flashinfer autotune by default (#14357) 2025-12-16 23:01:39 -08:00
Baizhou Zhang
d92c1f8cbd Fix lora doc (#15282) 2025-12-16 15:53:38 -08:00
amysaq2023
ccc8f3b266 support non disturbing remote instance weight loader v2 (#14997) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-16 14:39:56 -08:00
Tianyu Guo
a9a2cdd8ec Add EPD disaggregation doc (#15224) 2025-12-16 11:22:05 +08:00
Hanming Lu
e61dabf5e4 [Qwen3-next] support mamba radix cache for overlap scheduler (#14792) 2025-12-14 18:54:16 -08:00
Ho-Ren (Jack) Chuang
171b442ad3 Add KV4-capable backend flashmla and update server args (#14989) 
Signed-off-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
 2025-12-12 11:50:27 -08:00
Yineng Zhang
4b7b5af36a Revert several PRs (#14958) 
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-12-12 11:25:12 -08:00
Shangming Cai
c8cf1cafdb [1/N] Update doc of Pipeline Parallelism (#14985) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: alpha-baby <fujianhao1997@qq.com>
 2025-12-12 19:32:52 +08:00
Ho-Ren (Jack) Chuang
10146af099 Check KV4 compatibility with attention backends and add KV4 support to the attention_backend doc (#14467) 
Signed-off-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
 2025-12-11 19:00:53 -08:00
Xinyuan Tong
9975acf50f [refactor] Update reasoning parameter to require_reasoning (#14922) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 16:55:59 -08:00
amysaq2023
70758d457e support non-disturbing remote-instance-weight-loader (#13125) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-11 16:45:32 -08:00
fzyzcjy
d85fecb536 Tiny fix incorrect worker removal command (#14822) 2025-12-10 22:29:39 +08:00
b8zhong
55504df2f7 Add FP8 Blockwise GEMM Backend Flag --fp8-gemm-backend (#14379) 2025-12-09 12:05:56 -08:00
Muqi Li
06836ad02a [Reasoning + Structured Output] make reasoning compatible with structured output (#12551) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-08 01:28:39 -08:00
Binyao Jiang
cf0478d602 [Glm46v] Bug fix for accuracy drop and unable to launch server (#14585) 
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
 2025-12-07 23:45:02 -08:00
fzyzcjy
1915a1f8a7 Super tiny remove unneeded policy flag (#14608) 2025-12-07 19:35:07 -08:00
Zhiyu
f6423b626c Rename TensorRT Model Optimizer to Model Optimizer (#14455) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
 2025-12-07 13:18:20 -08:00
b8zhong
dd91d38e6a [Doc] Add short explanation on page size (#14557) 2025-12-06 17:26:30 -08:00
Feng Su
e12c6b320f [model-gateway][tracing]: implement request tracing using OpenTelemetry with trace context propagation (HTTP) (#13897) 2025-12-06 05:59:04 -08:00
Rain Jiang
ea177372bd support mtp with deepseek r1 nvfp4 model (#13115) 
Co-authored-by: Trevor Morris <tmorris@nvidia.com>
 2025-12-06 00:45:54 -08:00
Baizhou Zhang
42fcf5438f Revert "tiny remove deprecated endpoint call" (#14533) 2025-12-05 23:48:54 -08:00
Yuhao Yang
7b0c7ad163 Revise DP Multi-Modal Encoder Document (#14290) 2025-12-06 12:56:53 +08:00
b8zhong
ec7b2c16d9 tiny remove deprecated endpoint call (#13607) 2025-12-05 09:54:49 -08:00
b8zhong
88d1bab537 add doc for quantized kv cache (#14348) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
Co-authored-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
 2025-12-04 13:01:05 -08:00
b8zhong
65c8568c4a sync attention, deepseek doc (#14335) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-02 21:19:40 -08:00
Vikram
42271376d1 [bug fix] use npu phy id in container env (#14266) 
Co-authored-by: jinke15 <jinke15@jd.com>
 2025-12-03 11:33:43 +08:00
Baizhou Zhang
922054079c [Doc] Update DeepSeek-V3.2 document (#14321) 2025-12-02 18:19:39 -08:00
Yuan Luo
21ec99beff [VLM][Doc] Document for VLM DP Encoder (#14279) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-02 15:08:37 +08:00
b8zhong
e6420100ee sync attention doc and ep doc to doctree (#14257) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-01 21:15:22 -08:00
Lianmin Zheng
bc3d2a85af [Minor] update docs (#14212) 2025-12-01 02:33:58 -08:00
Wenyi Xu
5e70880e64 [model-gateway] Add PostgreSQL support to binding (#13766) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
Co-authored-by: Chang Su <chang.s.su@oracle.com>
 2025-11-26 09:11:13 -08:00
Liangsheng Yin
6c190cbda0 Rename: --hooks to --forward-hooks (#13994) 2025-11-26 22:26:28 +08:00
Cheng Wan
bf10869203 [Doc] Add an Introduction to Expert Parallelism (#13783) 2025-11-24 14:46:51 -08:00
Binyao Jiang
b29769f3b6 Move unnecessary input_addr capture under debug mode flag for speed-up (#13690) 2025-11-22 11:42:26 -08:00
赵晨阳
c56fc42430 Update quantization.md with new model resources (#13677) 2025-11-20 15:50:16 -08:00
Liangsheng Yin
196b940aed [3/N] CI refactor: move some manually triggered tests. (#13448) 2025-11-19 23:06:53 +08:00
ykwd
075ba74dd4 [Doc] Update HiCache and Mooncake docs & Mooncake Setup Error Checking (#12740) 2025-11-19 12:03:27 +08:00
wingedge
f1be8aa0f2 chore: add an unified server arg for multimodal inputs preprocess config(#12149) 
Co-authored-by: bianfeng <bianfeng@pinduoduo.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-11-18 12:18:50 +08:00
Binyao Jiang
90c18a16cb [GLM4.6v] Required changes for bumping up to transformer 5.x (#13229) 2025-11-18 10:58:00 +08:00
pansicheng
d879e37f1b Add interface_v1 option for dynamic HiCache backend (#13140) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-11-17 17:42:17 -08:00
Carlo Mussolini
df56139226 Adding user defined hooks support (#13217) 2025-11-17 23:07:37 +08:00
lixiaolx
d368c7451a (1/n)support context parallel with deepseekv3.2-DSA (#12065) 2025-11-16 20:12:25 -08:00
Lianmin Zheng
7e626d12b7 Update docs (#13391) 
Co-authored-by: sglang-bot <sglangbot@gmail.com>
 2025-11-16 19:36:33 -08:00
kyleliang-nv
597d416070 [feature] Add layerwise NVTX support (#11870) 2025-11-15 19:20:56 -08:00
Zhihao Lyu
10592e9c08 [Ascend][Feat] Add Ascend sampling backend (#12692) 
Co-authored-by: ronnie_zheng <zl19940307@163.com>
 2025-11-15 21:59:58 +08:00
Yongfei Xu
d91b16eb16 Opt tp: tp attn support tp reduce scattered input (#10568) 2025-11-15 02:08:12 -08:00
khalilzhk
2d531946db [Ascend][feature] support L1+ L2 radixcache on ascend (#12214) 
Co-authored-by: khalilzhk <zhangkaikai12@huawei.com>
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
 2025-11-12 20:45:24 +08:00
Mattheliu
c3bb348dad [Docs] fix dead links in multiple documentation pages (#12764) 2025-11-06 10:49:32 -08:00
b8zhong
32ec68faf9 keep attention backend document up to date (#12741) 2025-11-05 20:41:20 -08:00
Trevor Morris
dbcf85b7f0 Add --speculative-moe-runner-backend server arg (#10183) 2025-11-04 00:20:56 -08:00