chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
7,655 Commits 1 Branch 0 Tags
02f1e81e2defa70b5bcdc59eb878dec415fdfd12
Commit Graph

110 Commits

Author SHA1 Message Date
b8zhong
55504df2f7 Add FP8 Blockwise GEMM Backend Flag --fp8-gemm-backend (#14379) 2025-12-09 12:05:56 -08:00
Muqi Li
06836ad02a [Reasoning + Structured Output] make reasoning compatible with structured output (#12551) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-08 01:28:39 -08:00
Binyao Jiang
cf0478d602 [Glm46v] Bug fix for accuracy drop and unable to launch server (#14585) 
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
 2025-12-07 23:45:02 -08:00
fzyzcjy
1915a1f8a7 Super tiny remove unneeded policy flag (#14608) 2025-12-07 19:35:07 -08:00
Zhiyu
f6423b626c Rename TensorRT Model Optimizer to Model Optimizer (#14455) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
 2025-12-07 13:18:20 -08:00
b8zhong
dd91d38e6a [Doc] Add short explanation on page size (#14557) 2025-12-06 17:26:30 -08:00
Feng Su
e12c6b320f [model-gateway][tracing]: implement request tracing using OpenTelemetry with trace context propagation (HTTP) (#13897) 2025-12-06 05:59:04 -08:00
Rain Jiang
ea177372bd support mtp with deepseek r1 nvfp4 model (#13115) 
Co-authored-by: Trevor Morris <tmorris@nvidia.com>
 2025-12-06 00:45:54 -08:00
Baizhou Zhang
42fcf5438f Revert "tiny remove deprecated endpoint call" (#14533) 2025-12-05 23:48:54 -08:00
Yuhao Yang
7b0c7ad163 Revise DP Multi-Modal Encoder Document (#14290) 2025-12-06 12:56:53 +08:00
b8zhong
ec7b2c16d9 tiny remove deprecated endpoint call (#13607) 2025-12-05 09:54:49 -08:00
b8zhong
88d1bab537 add doc for quantized kv cache (#14348) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
Co-authored-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
 2025-12-04 13:01:05 -08:00
b8zhong
65c8568c4a sync attention, deepseek doc (#14335) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-02 21:19:40 -08:00
Vikram
42271376d1 [bug fix] use npu phy id in container env (#14266) 
Co-authored-by: jinke15 <jinke15@jd.com>
 2025-12-03 11:33:43 +08:00
Baizhou Zhang
922054079c [Doc] Update DeepSeek-V3.2 document (#14321) 2025-12-02 18:19:39 -08:00
Yuan Luo
21ec99beff [VLM][Doc] Document for VLM DP Encoder (#14279) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-02 15:08:37 +08:00
b8zhong
e6420100ee sync attention doc and ep doc to doctree (#14257) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-01 21:15:22 -08:00
Lianmin Zheng
bc3d2a85af [Minor] update docs (#14212) 2025-12-01 02:33:58 -08:00
Wenyi Xu
5e70880e64 [model-gateway] Add PostgreSQL support to binding (#13766) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
Co-authored-by: Chang Su <chang.s.su@oracle.com>
 2025-11-26 09:11:13 -08:00
Liangsheng Yin
6c190cbda0 Rename: --hooks to --forward-hooks (#13994) 2025-11-26 22:26:28 +08:00
Cheng Wan
bf10869203 [Doc] Add an Introduction to Expert Parallelism (#13783) 2025-11-24 14:46:51 -08:00
Binyao Jiang
b29769f3b6 Move unnecessary input_addr capture under debug mode flag for speed-up (#13690) 2025-11-22 11:42:26 -08:00
赵晨阳
c56fc42430 Update quantization.md with new model resources (#13677) 2025-11-20 15:50:16 -08:00
Liangsheng Yin
196b940aed [3/N] CI refactor: move some manually triggered tests. (#13448) 2025-11-19 23:06:53 +08:00
ykwd
075ba74dd4 [Doc] Update HiCache and Mooncake docs & Mooncake Setup Error Checking (#12740) 2025-11-19 12:03:27 +08:00
wingedge
f1be8aa0f2 chore: add an unified server arg for multimodal inputs preprocess config(#12149) 
Co-authored-by: bianfeng <bianfeng@pinduoduo.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-11-18 12:18:50 +08:00
Binyao Jiang
90c18a16cb [GLM4.6v] Required changes for bumping up to transformer 5.x (#13229) 2025-11-18 10:58:00 +08:00
pansicheng
d879e37f1b Add interface_v1 option for dynamic HiCache backend (#13140) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-11-17 17:42:17 -08:00
Carlo Mussolini
df56139226 Adding user defined hooks support (#13217) 2025-11-17 23:07:37 +08:00
lixiaolx
d368c7451a (1/n)support context parallel with deepseekv3.2-DSA (#12065) 2025-11-16 20:12:25 -08:00
Lianmin Zheng
7e626d12b7 Update docs (#13391) 
Co-authored-by: sglang-bot <sglangbot@gmail.com>
 2025-11-16 19:36:33 -08:00
kyleliang-nv
597d416070 [feature] Add layerwise NVTX support (#11870) 2025-11-15 19:20:56 -08:00
Zhihao Lyu
10592e9c08 [Ascend][Feat] Add Ascend sampling backend (#12692) 
Co-authored-by: ronnie_zheng <zl19940307@163.com>
 2025-11-15 21:59:58 +08:00
Yongfei Xu
d91b16eb16 Opt tp: tp attn support tp reduce scattered input (#10568) 2025-11-15 02:08:12 -08:00
khalilzhk
2d531946db [Ascend][feature] support L1+ L2 radixcache on ascend (#12214) 
Co-authored-by: khalilzhk <zhangkaikai12@huawei.com>
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
 2025-11-12 20:45:24 +08:00
Mattheliu
c3bb348dad [Docs] fix dead links in multiple documentation pages (#12764) 2025-11-06 10:49:32 -08:00
b8zhong
32ec68faf9 keep attention backend document up to date (#12741) 2025-11-05 20:41:20 -08:00
Trevor Morris
dbcf85b7f0 Add --speculative-moe-runner-backend server arg (#10183) 2025-11-04 00:20:56 -08:00
b8zhong
d31d48b341 update usage of trtllm_fp8_per_tensor_scale_moe (#12569) 2025-11-03 14:25:32 -08:00
b8zhong
a076ec1a7a Revert "fix llama4 kv cache layout" (#12437) 2025-10-30 22:33:37 -07:00
Teng Ma
32438eba45 [Ckpt Engine] feat: new sglang entrypoint support for update (#12216) 2025-10-30 10:39:27 +08:00
Weiwei
caa4819bfc Add support for AutoRound quantized models (#10153) 2025-10-27 18:17:29 +08:00
Xinyuan Tong
3f4cc0aff0 Remove description for --enable-beta-spec argument (#12177) 2025-10-26 16:49:39 -07:00
Baizhou Zhang
729b242934 [Doc] Add documentation for DeepSeek V3.2 (#11877) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: ybyang <ybyang7@iflytek.com>
 2025-10-24 19:06:22 -07:00
Jonah Bernard
4b046a72d3 docs(server-arguments): add allowed options for each argument (#11560) 2025-10-24 11:49:20 -07:00
Baizhou Zhang
983ef22cf3 [Doc] Update deterministic inference flag in server_arguments.md (#11978) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-22 14:12:15 -07:00
Minglei Zhu
200a3c0bb1 [Documentation] add doc for deterministic inference (#11956) 2025-10-22 12:36:15 -05:00
Zhiyu
80b2b3207a Enable native ModelOpt quantization support (3/3) (#10154) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
 2025-10-21 21:44:29 -07:00
Baizhou Zhang
ef4a8097b8 Rename flashmla kernel options of nsa backend for better readability (#11876) 2025-10-21 13:14:16 -07:00
Neelabh Sinha
852c0578fd [FEATURE] Add OpenAI-Compatible LoRA Adapter Selection (#11570) 2025-10-21 15:44:33 +08:00