chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
6,405 Commits 1 Branch 0 Tags
57cc5385c0c58c1925ab1da5050aedf987b72cc0
Commit Graph

768 Commits

Author SHA1 Message Date
b8zhong
a076ec1a7a Revert "fix llama4 kv cache layout" (#12437) 2025-10-30 22:33:37 -07:00
Lianmin Zheng
7cd716f776 Update news section in README.md (#12409) 2025-10-30 12:41:21 -07:00
Baizhou Zhang
621dfb8886 Import flash_mla from sgl-kernel (#12135) 2025-10-29 23:54:21 -07:00
Teng Ma
32438eba45 [Ckpt Engine] feat: new sglang entrypoint support for update (#12216) 2025-10-30 10:39:27 +08:00
Yuhong Guo
caa5d2967c feat: return partial generation results when aborting requests in waiting queue (#11673) 2025-10-29 22:03:00 +08:00
Yingchun Lai
5e36a0b455 [metrics][EPLB]: Support selected count of physical experts on each GPU (#9825) 2025-10-28 20:56:19 -07:00
hlu1
0ee831dee0 Update deepseek_v32.md (#12296) 2025-10-28 14:52:38 -07:00
ybyang
9c6e25d2a6 doc for logit_bias (#12188) 2025-10-28 10:32:12 -07:00
Feng Su
ea96106000 [Feature] Sglang Tracing: Fine-Grained Tracking for Request Latency - Part 2 (#10804) 
Signed-off-by: Feng Su <sufeng@linux.alibaba.com>
 2025-10-28 01:25:46 -07:00
ishandhanani
285a8e6986 docker: add CUDA13 support in dockerfile and update GDRCopy/NVSHMEM for blackwell support (#11517) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-10-27 22:00:54 -07:00
satyamk7054
9fc3e8aac7 Add support for Matryoshka embeddings (#126) (#11142) 
Co-authored-by: Satyam Kumar <satyamk@linkedin.com>
 2025-10-28 02:49:36 +08:00
Weiwei
caa4819bfc Add support for AutoRound quantized models (#10153) 2025-10-27 18:17:29 +08:00
sglang-bot
55d75e11bd chore: bump SGLang version to 0.5.4.post1 (#12169) 2025-10-27 09:35:20 +08:00
Xinyuan Tong
3f4cc0aff0 Remove description for --enable-beta-spec argument (#12177) 2025-10-26 16:49:39 -07:00
赵晨阳
7ebc28f5d6 [WIP] support MiniMax M2 model (#12129) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Signed-off-by: xuebi <xuebi@minimaxi.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Roger Young <42564206+rogeryoungh@users.noreply.github.com>
Co-authored-by: xuebi <xuebi@minimaxi.com>
 2025-10-26 13:58:54 -07:00
Zijian Zhang
a8023891f6 model: support NVILA and NVILA Lite (#10399) 2025-10-26 09:58:09 -07:00
Baizhou Zhang
97828878d8 [Doc] Small update of DeepSeek v3.2 document (#12138) 2025-10-25 20:34:05 -07:00
Vincent Zhong
b4d2da106e [docs] upd docker files names everywhere (#12133) 2025-10-25 18:20:23 -07:00
Baizhou Zhang
bcecf27e7c [Doc] Fix format for deepseek v3.2 document (#12130) 2025-10-25 15:07:50 -07:00
Baizhou Zhang
8e987fa2a3 Update document index for DeepSeek-v32 docs (#12101) 2025-10-25 13:38:58 -07:00
Lianmin Zheng
4caca1ba04 Clean up server args & Add CI scripts (#12124) 2025-10-25 11:53:57 -07:00
Kangyan-Zhou
ceb105a780 Update setup_github_runner.md 2025-10-25 09:22:00 -07:00
Baizhou Zhang
729b242934 [Doc] Add documentation for DeepSeek V3.2 (#11877) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: ybyang <ybyang7@iflytek.com>
 2025-10-24 19:06:22 -07:00
Jonah Bernard
4b046a72d3 docs(server-arguments): add allowed options for each argument (#11560) 2025-10-24 11:49:20 -07:00
ishandhanani
14203432b4 fix(compile_utils, ep_moe): update environment variable and dtype check (#12034) 2025-10-24 11:00:12 -07:00
Lianmin Zheng
ffc722a690 Revert "lang: support direct video inference" (#12038) 2025-10-23 19:21:31 -07:00
thelongestusernameofall
49afb3d9d9 Fix(security): block unsafe pickle deserialization to mitigate CVE-2025-10164 (#11909) 
Co-authored-by: Chengxing Xie <xiechengxing34@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-23 19:12:40 -07:00
Mick
823b442945 lang: support direct video inference (#9936) 
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
 2025-10-23 18:12:39 -07:00
sglang-bot
1053e1be17 chore: bump SGLang version to 0.5.4 (#12027) 
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
 2025-10-23 18:01:40 -07:00
Teng Ma
96a5e4dd79 [Feature] Support loading weights from ckpt engine worker (#11755) 
Signed-off-by: Yang Kaiyong <yangkaiyong.yky@antgroup.com>
Signed-off-by: Cruz Zhao <CruzZhao@linux.alibaba.com>
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Yang Kaiyong <yangkaiyong.yky@antgroup.com>
Co-authored-by: Cruz Zhao <CruzZhao@linux.alibaba.com>
Co-authored-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-10-23 09:23:30 -07:00
Zaili Wang
007b849b0e [CPU] misc updates (#11906) 2025-10-22 21:10:05 -07:00
Baizhou Zhang
983ef22cf3 [Doc] Update deterministic inference flag in server_arguments.md (#11978) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-22 14:12:15 -07:00
Minglei Zhu
200a3c0bb1 [Documentation] add doc for deterministic inference (#11956) 2025-10-22 12:36:15 -05:00
Zhiyu
80b2b3207a Enable native ModelOpt quantization support (3/3) (#10154) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
 2025-10-21 21:44:29 -07:00
Baizhou Zhang
ef4a8097b8 Rename flashmla kernel options of nsa backend for better readability (#11876) 2025-10-21 13:14:16 -07:00
ybyang
dbb16bedd5 Support Thinking Budget (via custom_logit_processor for OpenAI API) [Fix #6572] (#11416) 
Signed-off-by: ybyang <ybyang7@iflytek.com>
Co-authored-by: YorkSu <york_su@qq.com>
 2025-10-21 16:27:56 +08:00
Neelabh Sinha
852c0578fd [FEATURE] Add OpenAI-Compatible LoRA Adapter Selection (#11570) 2025-10-21 15:44:33 +08:00
Meng, Hengyu
b113c72e7a Init attention backend for Intel XPU (#10656) 
Co-authored-by: guangyey <guangye.yu@intel.com>
Co-authored-by: DiweiSun <105627594+DiweiSun@users.noreply.github.com>
 2025-10-21 11:41:28 +08:00
DarkSharpness
276e7b3e4e [Feature] New structural tag support (#10691) 2025-10-20 18:25:58 +08:00
Sai Enduri
e53bf44243 Update amd gpu install docs. (#11849) 2025-10-20 00:03:26 -07:00
Shane A
d383e6616e [Model] Add Olmo 3 model support (#11396) 2025-10-19 23:59:16 -07:00
Shangming Cai
a2ba0bc3df Tiny clean up for PD module and doc (#11747) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-10-20 11:52:42 +08:00
Baizhou Zhang
44f0ece9fc [Doc] Update documents for FA4 (#11778) 2025-10-19 17:40:38 -07:00
ybyang
b5e14b2b78 [1/2][feature] support openai like classification api (#11618) 2025-10-18 19:32:48 -07:00
b8zhong
f9a7d9b3dc support server arg override KV cache to bf16 to avoid slow cases (#11749) 2025-10-19 02:49:48 +08:00
Lianmin Zheng
67e34c56d7 Fix install instructions and pyproject.tomls (#11781) 2025-10-18 01:08:01 -07:00
Qiaolin Yu
547003bdd0 fix command line usage of profiling (#11793) 2025-10-18 12:54:36 +08:00
Lianmin Zheng
9eefe2c0b7 Set CUDA_VISIBLE_DEVICES to achieve one GPU per process (#9170) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Cheng Wan <cwan@x.ai>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-10-17 17:30:06 -07:00
Lianmin Zheng
b9a54e0968 [minor] sync code on python/sglang/test/test_deterministic.py and improve ci tests (#11777) 
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
 2025-10-17 14:25:22 -07:00
Keyang Ru
2bc3fcd420 [doc] update router document (#11767) 2025-10-17 10:26:54 -07:00